MP3 vs WAV: The Blind Test - Can YOU tell the difference?

[Mark]

This sounds like an interesting test! I have to drive to LA now and will probably work all night, so please don't disclose the answers before I get a chance to listen to the clips tomorrow!

No problem. You have until Sunday July 1st until I reveal the answers.


On a general note, thanks to all so far for taking part in this experiment. Your responses are ... interesting. If I said any more than that at this stage, I'd risk messing with the relative purity of the exercise.

Keep your guesses coming in.

[Mark]

Well, I've just checked the site stats for the Blind Test, and there have so far been 33 visitors. So come on: let's have a few more guesses.

[71 dB]

There are many people who appreciate high-quality sound. There are just as many who claim they can tell the difference between compressed and uncompressed digital sound files. So, I've decided we should put this to the test.

Click through to my hastily erected Blind Test Site, then click, one at a time, on the arrows beside each of the alphabetically listed clips. You'll eventually have downloaded (or heard immediately in your default media player) a total of four, identical two-minute extracts from the end of the fourth movement of Berlioz's 'Symphonie Fantastique' (VPO/Davis '90).

You'll notice each download is a 20.1Mb WAV file. But don't be fooled: three of these are NOT real WAV files. They were transcoded back into WAV files from a 128kbps MP3 file, a 192kbps MP3 file and a 320kbps MP3 file respectively to make this experiment more objective. (The three compression bitrates were chosen because these are the most commonly used.)

Listen to each clip in turn, then tell the forum which is the genuine uncompressed WAV file, and which clips are the 128kbps, 192kbps and 320kbps MP3 files. Post your guesses in this thread, and I'll post the correct answers on Sunday July 1st.

This is not a competition. It's an exercise to see how well those who choose to take part can detect compression artifacts. And yes, do feel free to use whatever technology you have at your disposal to get the clearest possible playback before posting your guesses.

As a thank you for taking part in this experiment, I'll upload (in about a week's time) the complete recording from which this test was drawn.

Good luck!


IMPORTANT NOTE: Anyone deciding to try and cheat by looking at the exact file size of each clip and concluding that largest equals WAV is in for a surprise. When transcoding in the way that I have done for this test, it often happens that the converted MP3 files end up LARGER than the original WAV files. So be on your guard ... it's not as obvious as you think.

Interesting test Mark! I am dowloading the files (the samples could have been shorter for smaller filesize).

[Mark]

Interesting test Mark! I am dowloading the files (the samples could have been shorter for smaller filesize).

Had to make them two minutes in length, 71dB, as I needed sufficient breadth of dynamics and complexity of sound to show off (or show up) the differences between the files.

[Mark]

Incidentally, I may well run this test again - using a different clip - pitting AAC against WAV. People tell me that AAC is 'better' than MP3. I'd love to put that to the test.

Might even do a blind listening test between AAC and MP3 ... watch this space.

[71 dB]

I can't tell the difference & I'd be skeptical if anyone tried to say that they can. It's because when mp3s are created only the frequencies inaudible to humans get cut out.

Frequencies mask other (merely higher) frequencies in human hearing. A weak 500 Hz tone can't be heard together with a loud 400 Hz tone. Masking happens in time to. Immediately after a strong sound the ability to hear weak sounds is reduced. Masking is very complex phenomena and mp3 coding only simulates it with simplified rules. The signal itself determines what frequencies in it can't be heard. Even the volume level affects this. Mode desibels means more masking.

So, the coding artifacts are heard when the simulated masking model differs from actual masking or when there simply isn't enough bits to conserve all audible information (low bitrates).

[Mark]

In case anyone's even remotely interested, I ripped the WAV sample clip using EAC, then used AbyssAudio's converter transcoding software (which uses Blade for MP3 encoding) to create the MP3 files and the subsequent fake WAVs. Why am I tell you this? Because anyone who knows of EAC's reputation for high-quality, highly accurate ripping will know that the WAV file in this test is about as close to the original as it was possible to get.

[71 dB]

I can't hear the difference between these clips.
I should be able to tell what is 128 kbps but I can't. Embarassing! 

I have a feeling that A is 128 kbps and D is original but I can be totally wrong.

I found the original sound quality low. A better sounding recording might make the differencies more clear.

[Holden]

This download site is sooo sloooooow!!!!!

[Mark]

This download site is sooo sloooooow!!!!!

Yes, sorry about that. The site is hosted on a server that doesn't give me a great deal of bandwidth. Maybe that's the problem? The samples download like lightning for me, however.

I can't hear the difference between these clips. I should be able to tell what is 128 kbps but I can't. Embarassing! 

And you've got super-high-quality equipment, yes?

[71 dB]

And you've got super-high-quality equipment, yes?

Crappy soundcard connected digitally to my NAD T762 AV-amplifier. Listening with Sennheiser HD580 phones.

[Mark]

Crappy soundcard connected digitally to my NAD T762 AV-amplifier. Listening with Sennheiser HD580 phones.

Try burning the clips to a CD-R then listening through your main hifi set-up.

[71 dB]

Try burning the clips to a CD-R then listening through your main hifi set-up.

Too much trouble. My soundcard is crappy but the connection is digital.

[Mark]

I found the original sound quality low. A better sounding recording might make the differencies more clear.

This was also deliberate. If I'd gone for a modern recording in startlingly terrific sound (like the CD layer of an SACD disc), the experiment wouldn't have been as representative of the many back catalogue downloads commercially available. What I'm trying to ascertain here is whether or not such classical downloads are at least good enough for the majority of discerning listeners. This blind test is therefore an attempt by me to replicate 'real world' conditions as much as I'm able to.

[Mark]

I suppose this brings me on to the precise reason why I chose the three bitrates that I did.

iTunes Music Store gives you 128kbps files by default (in AAC, I know ... we'll come to that test in time ).

eclassical (and many others) gives you 192kbps files.

Classics & Jazz (Universal's site) gives you 320kbps files.

Linn Records (and a few others, I'm told) gives you uncompressed files.

It's long intrigued me to know whether or not people really hear the differences in these varying bitrates. Or maybe we just assume something is better because we're told so. In some respects, this test mirrors M Forever's Mystery Orchestra challenge. When you can't see the label, can you tell the difference? That's what interests me.

[71 dB]

This was also deliberate. If I'd gone for a modern recording in startlingly terrific sound (like the CD layer of an SACD disc), the experiment wouldn't have been as representative of the many back catalogue downloads commercially available. What I'm trying to ascertain here is whether or not such classical downloads are at least good enough for the majority of discerning listeners. This blind test is therefore an attempt by me to replicate 'real world' conditions as much as I'm able to.

I see. In real life I avoid older recordings.

Old noisy recordings can be even more difficult than new noise-free recordings. Noise is always poison for coders.

[Haffner]

And yet, there are so many people who confidently claim that they can hear the difference between compressed and uncompressed sound when comparing these common MP3 file bitrates to source WAV files.

This Rock Star person, Kelly Keeling, agreed to sing a couple of songs for my project last year. The straw that broke the camel's back was that he asked me to send him backing tracks to sing on , quote "32-bit WAV by 5:oo" the very same day. That was a little too Rock Star for me and I accidentally/on purpose lost his phone number.

[The Mad Hatter]

Have listened to them...I have my ideas, but I'm going to listen to them again before I post anything definite.

[M forever]

Well, I've just checked the site stats for the Blind Test, and there have so far been 33 visitors. So come on: let's have a few more guesses.

Welcome to the world of blind test hosting. Some of my Mystery Orchestra samples over on RMCR were downloaded as much as 50-70 times, yet less than a dozen people regularly posted.

*None* of the loudmouths with the most extreme and rigid...opinions...ever posted except sometimes just to say that they thought the whole idea was idiotic. That included several people who were very ardent admirers and "outspoken" supporters of "Joyce Hatto", including one guy who had actually reviewed several "Hatto recordings" and declared them absolutely marvelous and who had also reviewed the same recording when it came out under the true artists' names, and totally trashed them 

Of course, that guy still posts the most specific "opinions" there, but he never played Mystery Orchestra to prove that he could actually hear what he is talking about. Not that it would have yielded any respectable results if he had. He once posted a lengthy piece about Abbado's recording of Bruckner 7 with the BP, and how the orchestra now sounded different from Karajan's time and this and that and a lot of very "specific" comments about the BP. Except that the orchestra on the disc is actually the WP...oops.

Anyway, you have to make clear that you can only see the *number*  of downloads, not *who* downloaded the clips, otherwise far fewer people will download them, because of paranoia.


Speaking of paranoia...I really don't like the way your new picture stares at me!

[Mark]

Speaking of paranoia...I really don't like the way your new picture stares at me!

And yes, you're right: I can't tell who is downloading what, only the number of unique visitors to the test site ... which currently stands at 44.

Looking forward to more guesses, people.