Windows: Lots of videos have built-in thumbnails in Windows Explorer, so you can see a quick preview of each file, but this only works with popular video formats that work out-of-the-box in Windows. Media Preview is a small utility that adds preview thumbnails to nearly every video format you can think of.
Media Preview supports 3GP, FLV, MKV, OGG and lots of other video formats that Windows doesn't recognise out of the box. Just install the plugin, select all the boxes on the configuration page and click Apply. You'll see that all of your video files change from their boring icons to pretty thumbnails. It's small, simple and does exactly what it says on the tin.
Media Preview is a free download for Windows only.