Windows only: If you find it troubling to stop running applications, clear out RAM, update drivers and otherwise get your system ready for a gaming session, free app Game Booster will do it all for you, according to your preferences.
Game Booster gives you a number of preferences for when you're running a game, from which background applications can quit to small tweaks to optimise processor performance and clean RAM. After specifying your preferences, you can go into "Gaming mode" with the flip of a switch. Then, when you're done playing, you can just switch gaming mode back off and go back to your regular desktop experience.
It also has a few other features, like checking for important driver updates, downloading popular game add-ons, and defragmenting your game folders. If you find that your desktop setup isn't optimal for gaming, it's a nice tool to let you spend more time gaming and less time tweaking your computer before every session. And, of course, don't forget to overclock your graphics card with previously mentioned EVGA Precision for an even bigger performance boost.
Game Booster 2 Beta is a free download, Windows only.