When your hard drive's cluttered with hundreds of files in folders scattered everywhere, chances are you're using up disk space with multiple copies of the same data. Whether multiple memory card downloads have plagued your photo library, or swapping MP3s with friends who have similar tastes left you with double copies of the same songs, or you've saved and re-saved the same documents in different places, a good file de-duping can recover hard drive space and help you get organized. We've featured several free tools for de-duplicating your documents, music, pictures and more over the years here at Lifehacker. Have a quick rundown so you can declutter your hard drive today.
All files—The open source WinMerge can identify duplicate images, text, and office documents in an intuitive interface. We gave WinMerge the full how-to treatment a while back, here's how to compare and merge files and folders with WinMerge.
If WinMerge's "open source look" turns you off, there are several other arguably more user-friendly options. DoubleKiller (original post) can scan more than two folders at once to identify duplicates based on criteria you set. Likewise, donationware Easy Duplicate Finder (original post) can compare more than two folders as well. Donationware CloneSpy (original post) compares file checksums in multiple folders to id dupes.
Microsoft Outlook—If it's a stuffed address book that contains repeats of the same contacts in Outlook that's bothering you, here's how to quickly delete duplicate Outlook contacts. Similarly, the Outlook Duplicate Items Remover (original post) can de-dupe your Outlook Contacts, Calendar, Tasks, Notes, and Email folders.
iTunes comes with a hidden "Show Duplicates" feature (go ahead, choose it from the View menu), but frankly, it kind of sucks. It does indeed show duplicates that you can flag for deletion, but it does it based on artist and song name—so if you've got a live version of the same song that's a completely different length and different album, iTunes will flag it. Lame.
I've tried a couple other free iTunes de-duping utilities (like Fixtunes), and nothing's worked very well (Fixtunes hung for hours with no indication of progress before I quit it). Shout out your favourite iTunes de-duper in the comments. (Note you can always use a regular file-de-duper and point it at your iTunes library instead.)
Bookmarks—If your bookmarks list is where your duplicate clutter is at, you want the Duplicate Detector Firefox extension (original post), which lets you search your existing bookmarks, ID and delete duplicates, and will warn you when you try to bookmark a URL already in your list. The Duplicate Detector is sadly not yet Firefox 3-compatible.
Mac OS X
Otherwise, there isn't the surfeit of free de-duplication utilities for Mac like there is for Windows, besides a couple command line scripts. Mac users, what do you use to scan and de-dupe the ol' Macintosh HD?
Got any other tips on how to clean up a big file mess of duplicate files? How do you declutter your hard drives? Let us know in the comments.