Not everyone can jump into the guts of their browser and tweak it using custom-made extensions or add-ons. While it gives you the most power, you have to get familiar with your browser's way of doing things, not to mention figuring out the API. At least for Firefox, you can try UI Eraser, which lets you surgically hide user interface elements.
Available at Add-ons for Firefox, UI Eraser is supported on version 29 and greater (though a few comments state that it's not working correctly from 35+). Once installed, you must click on the extension's icon, which will activate its design mode.
From here, you can delete any element it can identify or, if it supports colours, tweak them to your heart's content.
Sure, it doesn't give you a lot of depth when it comes to tailoring your Firefox experience, but if there are a couple of annoying UI bits you could do without, it doesn't get simpler than this.