The Great MP3 Bitrate Experiment

Lately, I've been trying to rid my life of as many physical artefacts as possible. I'm with Merlin Mann on CDs: "Can't believe how quickly CDs went from something I hate storing to something I hate buying to something I hate merely existing."

Although I'd extend that line of thinking to DVDs as well. The death of physical media has some definite downsides, but after owning certain movies once on VHS, then on DVD, and finally on Blu-ray, I think I'm now at peace with the idea of not owning any physical media ever again, if I can help it.

My current strategy of wishing my physical media collection into a cornfield involves shipping all our DVDs to Second Spin via media mail, and paying our nephew $1 per CD to rip our CD collection using Exact Audio Copy and LAME as a summer project. The point of this exercise is absolutely not piracy; I have no interest in keeping both digital and physical copies of the media I paid for the privilege of owning temporarily licensing. Note that I didn't bother ripping any of the DVDs because I hardly ever watched them; mostly they just collected dust. But I continue to love music and listen to my music collection on a daily basis. I'll donate all the ripped CDs to some charity or library, and if I can't pull that off, I'll just destroy them outright. Stupid atoms!

CDs, unlike DVDs or even Blu-rays, are considered reference quality. That is, the uncompressed digital audio data contained on a CD is a nearly perfect representation of the original studio master, for most reasonable people's interpretation of "perfect", at least back in 1980. So if you paid for a CD, you might be worried that ripping it to a compressed digital audio format would result in an inferior listening experience.

I'm not exactly an audiophile, but I like to think I have pretty good ears. I've recommended buying $US200+ headphones and headphone amps for quite a while now. By the way: still a good investment! Go do it! Anyhow, previous research and my own experiments led me to write Getting the Best Bang for Your Byte seven years ago. I concluded that nobody could really hear the difference between a raw CD track and an MP3 using a decent encoder at a variable bit rate averaging around 160kbps. Any bit rate higher than that was just wasting space on your device and your bandwidth for no rational reason. So-called "high-resolution audio" was recently thoroughly debunked for very similar reasons.

Articles last month revealed that musician Neil Young and Apple's Steve Jobs discussed offering digital music downloads of 'uncompromised studio quality'. Much of the press and user commentary was particularly enthusiastic about the prospect of uncompressed 24 bit 192kHz downloads. 24/192 featured prominently in my own conversations with Mr. Young's group several months ago. Unfortunately, there is no point to distributing music in 24-bit/192kHz format. Its playback fidelity is slightly inferior to 16/44.1 or 16/48, and it takes up 6 times the space. There are a few real problems with the audio quality and 'experience' of digitally distributed music today. 24/192 solves none of them. While everyone fixates on 24/192 as a magic bullet, we're not going to see any actual improvement.

The authors of LAME must have agreed with me, because the typical, standard, recommended, default way of encoding any old audio input to MP3 now produces variable bit rate MP3 tracks at a bitrate of around 192kbps on average.

(Going down one level to the "medium" preset produces nearly exactly 160kbps average, my 2005 recommendation on the nose.)

Encoders have only gotten better since the good old days of 2005. Given the many orders of magnitude improvement in performance and storage since then, I'm totally comfortable with throwing an additional 32kbps in there, going from 160kbps average to 192kbps average just to be totally safe. That's still a minuscule file size compared to the enormous amount of data required for mythical, aurally perfect raw audio. For a particular four-minute and 56-second music track, that'd be:

Uncompressed raw CD format 51MB Lossless FLAC compression 36MB LAME insane encoded MP3 (320kbps) 11.6MB LAME standard encoded MP3 (192kbps avg) 7.1MB

Ripping to uncompressed audio is a non-starter. I don't care how much of an ultra audio quality nerd you are, spending 7× or 5× the bandwidth and storage for completely inaudible "quality" improvements is a dagger directly in the heart of this efficiency-loving nerd, at least. Maybe if you're planning to do a lot of remixing and manipulation it might make sense to retain the raw source audio, but for typical listening, never.

The difference between the 320kbps track and the 192kbps track is more rational to argue about. But it's still 1.6 times the size. Yes, we have tons more bandwidth and storage and power today, but storage space on your mobile device will never be free, nor will bandwidth or storage in the cloud, where I think most of this stuff should ultimately reside. And all other things being equal, wouldn't you rather be able to fit 200 songs on your device instead of 100? Wouldn't you rather be able to download 10 tracks in the same time instead of 5? Efficiency, that's where it's at. Particularly when people with dog's ears wouldn't even be able to hear the difference.

But Wait, I Have Dog Ears

Of course you do. On the internet, nobody knows you're a dog. Personally, I think you're a human being full of crap, but let's drop some science on this and see if you can prove it.

When someone tells me, "Dudes, come on, let's steer clear of the worst song ever written!", I say challenge accepted. Behold The Great MP3 Bitrate Experiment!

As proposed on our very own Audio and Video Production Stack Exchange, we're going to do a blind test of the same two-minute excerpt of a particular rock audio track at a few different bitrates, ranging from 128kbps CBR MP3 all the way up to raw uncompressed CD audio. Each sample was encoded (if necessary), then exported to WAV so they all have the same file size. Can you tell the difference between any of these audio samples using just your ears?

Listen to each two minute audio sample Limburger Cheddar Gouda Brie Feta

Rate each sample for encoding quality

Once you've given each audio sample a listen — with only your ears please, not analysis software — fill out this brief form and rate each audio sample from one to five on encoding quality, where one represents worst and five represents flawless.

Yes, it would be better to use a variety of different audio samples, like SoundExpert does, but I don't have time to do that. Anyway, if the difference in encoding bitrate quality is as profound as certain vocal elements of the community would have you believe it is, that difference should be audible in any music track. To those who might argue that I am trolling audiophiles into listening to one of the worst-slash-best rock songs of all time … over and over and over … to prove a point … I say, how dare you impugn my honour in this manner, sir. How dare you!

I wasn't comfortable making my generous TypePad hosts suffer through the bandwidth demands of multiple 16 megabyte audio samples, so this was a fun opportunity to exercise my long dormant Amazon S3 account, and test out Amazon's on-demand CloudFront CDN. I hope I'm not rubbing any copyright holders the wrong way with this test; I just used a song excerpt for science, man! I'll pull the files entirely after a few weeks just to be sure.

You'll get no argument from me that the old standby of 128kbps constant bit rate encoding is not adequate for most music, even today, and you should be able to hear that in this test. But I also maintain that virtually nobody can reliably tell the difference between a 160kbps variable bit rate MP3 and the raw CD audio, much less 192kbps. If you'd like to prove me wrong, this is your big chance. Like the announcer in Smash TV, I say good luck — you're gonna need it.

So which is it — are you a dog or a man? Give the samples a listen, then rate them. I'll post the results of this experiment in a few days.

The Great MP3 Bitrate Experiment [Coding Horror]

Jeff Atwood is a blogger at Coding Horror and cofounder of Stack Overflow. Follow him on Twitter @codinghorror.


    Inexpensive headphones are one thing, but if you want to listen on a half-reasonable hifi in your lounge, or play the songs at volume, FLAC/ALAC/CDA is always going to be better. Something odd happens to the stereo image with compressed music, it's barely noticeable on headphones or low volumes, but you can tell "something's not quite right" when you play it on a bigger system.

      Move those WAVs onto your high end gear and post your votes up. I'm pretty sure if you can get the "not quite right" and "quite right" distinction correct (without cheating using software) then Jeff will be impressed.

    Each to their own. Telling everyone that anything above 160kbit is a waste of time is as arrogant as me telling you that everyone must encode in FLAC because I CAN pick the difference ... easily. It wasn't that long ago that 128kbit was considered almost unanimously as "CD quality" - which today is considered laughable, except by those that are noticeably tone deaf.

    Last time I tested blindly, I used cheap speakers through my PC and sat across the room and still picked the difference between 192 and 224 kbit samples... It reminds me of the difference of going from a 15in monitor to a 17in monitor ... there's little difference. But, try going from a 17in to a 15in monitor ... the difference is amazing... If my collection is encoded in a lower quality, it's hard to pick better quality when I hear it by exception.

    I keep my collection in FLAC, mainly because the extra space required is trivial - and the cost of the disk space is negligible. There are a number of issues with encoding in lossy format (besides being non-recoverable) and that list includes the loss of stereo sound in some circumstances,

    A little tip - if you find yourself getting a headache after listening to music for 1/2 an hour or so, try to encode the music at a higher quality. I didn't notice it at first - until a friend of mine (a sound engineer) pointed this out to me. I was listening to mp3s from 160 to 192kbits back then.

    James 2nd

      Well if you can hear the difference then prove it - post your assessment in advance of the results.

    Why not rip to FLAC or another lossless format, and then convert to MP3? That way, when you decide to change the format of your library (like you did between 2005 and now), you don't have to rip everything from the physical CDs again.

      This is exactly what I have done, and I have to say it's been a really great outlet for my anal-retentive tendencies - developing a consistent scheme for track numbers, "featured" artists, etc; getting all that metadata in there including album art; organising the actual file and folder structure... and the project continues, as there are a few things I missed the first time around.

      But since all of this is done in FLAC, I can dump it all into a [email protected] MP3 transcoder for my current needs; later on perhaps AAC or some other format will rise to prominence (or work better on my then-device of choice).

    "The point of this exercise is absolutely not piracy; I have no interest in keeping both digital and physical copies of the media I paid for the privilege of owning temporarily licensing. "

    "I’ll donate all the ripped CDs to some charity or library..."

    Erm, these sentences are mutually exclusive.
    If you're going to rip your licenced disk copy and not pirate, then you can't donate those disks to charity or a library, unless it's as an art project and the disks have been rendered unreadable... otherwise you've turned the single licence into two licences.. which is the whole issue with piracy.

    Oops, something went wrong.
    The form "Which MP3 audio sample has the highest and lowest quality?" is no longer accepting responses.
    Try contacting the owner of the form if you think this is a mistake

Join the discussion!