MP3 vs WAV: The Blind Test - Can YOU tell the difference?

[Mark]

There are many people who appreciate high-quality sound. There are just as many who claim they can tell the difference between compressed and uncompressed digital sound files. So, I've decided we should put this to the test.

Click through to my hastily erected Blind Test Site, then click, one at a time, on the arrows beside each of the alphabetically listed clips. You'll eventually have downloaded (or heard via the site's mini players) a total of four, identical two-minute extracts from the end of the fourth movement of Berlioz's 'Symphonie Fantastique' (VPO/Davis '90).

You'll notice each download is a 20.1Mb WAV file. But don't be fooled: three of these are NOT real WAV files. They were transcoded back into WAV files from a 128kbps MP3 file, a 192kbps MP3 file and a 320kbps MP3 file respectively to make this experiment more objective. (The three compression bitrates were chosen because these are the most commonly used.)

Listen to each clip in turn, then tell the forum which is the genuine uncompressed WAV file, and which clips are the 128kbps, 192kbps and 320kbps MP3 files. Post your guesses in this thread, and I'll post the correct answers on Sunday July 1st.

This is not a competition. It's an exercise to see how well those who choose to take part can detect compression artifacts. And yes, do feel free to use whatever technology you have at your disposal to get the clearest possible playback before posting your guesses.

As a thank you for taking part in this experiment, I'll upload (in about a week's time) the complete recording from which this test was drawn.

Good luck!


IMPORTANT NOTE: Anyone deciding to try and cheat by looking at the exact file size of each clip and concluding that largest equals WAV is in for a surprise. When transcoding in the way that I have done for this test, it often happens that the converted MP3 files end up LARGER than the original WAV files. So be on your guard ... it's not as obvious as you think.

[Justin Ignaz Franz Bieber]

I can't tell the difference & I'd be skeptical if anyone tried to say that they can. It's because when mp3s are created only the frequencies inaudible to humans get cut out.

[Mark]

I can't tell the difference & I'd be skeptical if anyone tried to say that they can.

And yet, there are so many people who confidently claim that they can hear the difference between compressed and uncompressed sound when comparing these common MP3 file bitrates to source WAV files.

[Mr. Darcy]

Fun challenge!

I confess, I really wanted to be able to discriminate between the compressed files (and the WAV file). I listened very closely on some mediocre Sony headphones. I'd like to think that someone with a better ear than I who can play these files at higher volumes on a decent setup can tell the difference but, I admit, I'm pretty much stumped. This was humbling..

However, I would like to hazard a guess about Clip D: I think I can detect some compression artifacts in there. So, I'm going to go with Clip D = 128 kbps MP3.

I'm prepared to be surprised and humiliated and I look forward to the results!

[Dancing Divertimentian]

It may all boil down to what type of system a person uses for playback.

I own a nice, mid-fi level home stereo system with a pair of speakers known for their high resolution, with clarity to burn (Revel M20's).

On my hi-res system noticing details is inescapable (and I wouldn't have it any other way). Which makes spotting such things as 'compressed' sound easy to detect.

To date I've only compared non-WAV burns to original CD's (nothing downloaded) and the difference is unquestionably there. Compressed burns have a distinctly 'off' sound that in reality might go undetected when listened to in isolation but once a direct comparison is made to the original CD the differences are readily apparent.

I find the compressed burns to be "fuzzier", less crisp, with some of the richer sound image from the original CD missing.

Whether or not I'd notice this on an entry level system I don't know but at least by mid-fi the differences are audible.

So if I ever did take to downloading you can bet it'd be WAV or some other lossless file.

[Mark]

All of which begs the question, Donwyn: Will you take this challenge using your system? Remember, one of the four samples was taken directly from the source CD - so, no compression.

Give it a go. I'd love to know if you can spot the fakers from the real deal in this test.

[Mark]

However, I would like to hazard a guess about Clip D: I think I can detect some compression artifacts in there. So, I'm going to go with Clip D = 128 kbps MP3.

Thanks for your guess. Care to guess at the others?

[George]

I did this using some decent Bose computer speakers and I am certain of all but which is the WAV and which is the highest bitrate MP3:

Here's what I got:

128 - D, definitely

192 - A, I think

320 - B, probably (it was pretty damn close)

WAV - C, I am pretty sure.


Mark what work is this BTW? I like it!

[George]

All of which begs the question, Donwyn: Will you take this challenge using your system? Remember, one of the four samples was taken directly from the source CD - so, no compression.

You didn't mess with the recording levels either, right Mark?

[Mark]

Mark what work is this BTW? I like it!

... two-minute extracts from the end of the fourth movement of Berlioz's 'Symphonie Fantastique' (VPO/Davis '90).

[George]

Thanks! I just changed my choice, BTW and I am now fairly certain of all four.

[Mark]

You didn't mess with the recording levels either, right Mark?

Didn't do a thing except compress and transcode back.

[George]

Didn't do a thing except compress and transcode back.

Just wanted to make sure. 

[Dancing Divertimentian]

All of which begs the question, Donwyn: Will you take this challenge using your system? Remember, one of the four samples was taken directly from the source CD - so, no compression.

Give it a go. I'd love to know if you can spot the fakers from the real deal in this test.

All right.

Lemme get back to ya...

[Mr. Darcy]

Thanks for your guess. Care to guess at the others? Smiley

I wouldn't dare...

[tjguitar]

I can't tell the difference & I'd be skeptical if anyone tried to say that they can. It's because when mp3s are created only the frequencies inaudible to humans get cut out.

What he said.

[PSmith08]

I would be inclined to agree with the statement that the MPEG-1 Audio Layer 3 codec (or any decent "lossy" codec) is designed to compress in such a way that you shouldn't notice the losses. I might, however, say that I can occasionally detect what I'll just have to call a loss of "bloom" with MP3/AAC tracks below a certain bitrate (~256 kbps). There's nothing "wrong" with the sound, except that it is a little duller and flatter than the CD. That could be a lot of things, but I think compression has to play a part.

I'm not going to try to tell. I have to practice a bit before I can get tin ears. 

[beclemund]

I could not manage to complete a download, but I would be interested in trying. I will check again later.

I agree with donwyn, though, playback would vary greatly depending on the system used. Unfortunately, I can only listen through my laptop or iPod in which case, compression quality makes even less of a difference for me.

[M forever]

This sounds like an interesting test! I have to drive to LA now and will probably work all night, so please don't disclose the answers before I get a chance to listen to the clips tomorrow!

[beclemund]

Ok, I managed to download them all finally... it was either my wireless connection or competing with other GMG'ers for your bandwidth. You are correct, Mark, this is difficult. I would guess that someone listening passively would have no trouble listening to a collection of 128kb MP3s and be satisfied with the experience. I did learn a few things in doing this test, however...

1) My Grado SR60 purchase a few months ago was a real waste. I could not detect any difference between any of the clips from my laptop. My Etymotics do a better job of reproducing highs, so I switched to them for the test (and frustrated my wife to no end as I could not hear her while trying to figure these out )

2) My iPod sounds much better than my laptop (tho' I already suspected that since I am sure the onboard audio solution on the laptop was of little consequence to the manufacturer where as the audio chip on the iPod is of high import to Apple).

3) Listening to four 2 minute segments back to back is too much... midway through the second clip I had already forgotten the first, so I had to force myself to scan to the same short segment in each clip... I wanted a cymbal crash, so forwarding to about the first 15 to 20 seconds and playing to 30 seconds gets two good ones in there. Also, I had to pick one clip as a baseline--I picked D.

4) Despite intent listening, I am still probably completely wrong and even if you try to actively listen to your music now and again, I doubt you will be disappointed by high quality lossy formats. My listening collection is in variable bit rate MP3, so you get the best of both worlds in terms of highest compression when you need it versus highest quality sound when you want it.

Ok, so here are my guesses, but the best I could do is break them into two groups.

Clips A and D: either 128 or 192kb I really could not distinguish between the two...
Clips B and C: 320kb or the WAV... I tried B and C back to back and like A and D, there was no difference between the two.

My first comparison was A to D and I knew I was in for a difficult test as they sound very much alike. Both B and C, however, sound much clearer than D. If I did not know this was a test going in, I probably would not have had any issues listening to either clip A or D independently though.

Again though, I am on pretty limited equipment so users of great equipment will probably have less of a challenge (and will probably be able to tell me if it's B or C that is the WAV file). There is a high end audio store not far from my house... maybe a burn to CD and test on their equipment would be in order.