Guybrush

EDIT : RESULTS ARE IN


Bit Rate Challenge No. 1


This thread is a spinoff thread from this one : http://www.musicbanter.com/general-m...mparisons.html

Some .. No wait - lemme start again. Many people claim that they are able to identify lower bitrate files from higher bitrate files. Some people suspect it, but perhaps they haven't tried. I'm giving everyone a chance to try this out now.

I've made a little html page with a flash player on it that plays mp3s. I guess most have flash, so I'm assuming there's no compatibility issue. What I want you to do is to listen to the mp3s on my flash player. It has a playlist on it with 4 files in different bitrates. All files are of course recordings of the same piece of music and they are named track 1 to 4. I want you to arrange track 1 to 4 as a sequence of 4 numbers starting with the lowest bitrate file and ending with the highest.

>> Test Yourself Here! << Edit : GONE!

The test is anonymous - noone except me will know your results unless you choose to share them!



PS! DO NOT POST YOUR RESULTS HERE AND DO NOT DISCUSS WHICH OF THE FILES YOU THINK IS WHICH! Instead, send your results to me by PM!

Please also say something about what equipment you used (pc speakers, earphones, hi end hi-fi system) and rate your own hearing ability (bad, medium, good).


I want each person who does this to be independent of others results and any discussion here. You can of course discuss non-results related stuff here.
This is round 1 and I may try other genres and so on later. Lemme know what you think.
Oh - and should you find a way to cheat, don't do it!


Details and methods :

The highest possible quality FLAC (lossless) was ripped from original Audio CD. The FLAC file was then opened in Audacity 1.2.6 and cut down to a ~30 seconds segment. This segment was then exported to MP3s of different bitrates using Lame MP3 Writer Plugin for Winamp v1.6.4 (lame_enc.dll).

__________________
Something Completely Different

Guybrush

Results :

I'll elaborate on these results later. I got you all in my excel sheet, but I'll keep an update going here. I'm giving each participant a number, then they know who they are but others don't You can then look up your own results here!

Listed are the amount of correct answers. Divide by four and add 100 and you have a percentage if you want.

1. 1
2. 0
3. 2
4. 0
5. 1
6. 1

__________________
Something Completely Different

lucifer_sam

i'd partake but i get some nasty noise from my cans because i don't have an outboard sound card to clean up my signal.

__________________
first.am

Guybrush

No submitted answers yet .. Come on guys!

If you're thinking you're not very good at it, it's not supposed to be easy. I suck at this myself

edit :

Okay, getting a few participants. I've put your participant numbers and scores up on the "board"

__________________
Something Completely Different

Guybrush

Guess I can post a little stats trivia since not much is happening in the thread

The chance of getting one answer right is 0,25 so the chance of getting everything right purely by chance is :

p(everything right)=0,25*0,25*0,25*0,25

.. which is of course miniscule. Even getting two answers right should be hard purely by chance.

__________________
Something Completely Different

music_phantom13

When do we get to see the answers? And has anyone else tried?

SATCHMO

I have to say that this test seems flawed in some ways. First of all the increments between bit-rates are, before any other factors are taken into consideration, too small. If the test were a blind comparison between two samples of say 196 and 320 it would be incredibly easier to tell the distinction, but 4 samples of varying bitrates using a player that allows the user only a minimal amount of control when conducting side by side comparisons to narrow down the bitrates( eg. you can compare sample 1 with sample 2 fairly easily, but not adequately compare sample 1 to sample 3 without a lot of maneuvering with the player).

Another thing that should be pointed out is the choice of music that was used for the sample. It's been my experience that the type of music that most clearly demonstrates shortcomings in bitrate performance are those recording that have both very quick transient attacks (aka crescendos). and dense recordings that place a lot of demands on dynamic headroom (mostly agressive heavy metal). I have many albums that are at 196kbps that I'm completely happy with because the peaks of the recordings dynamic range fall easily within the parameters of that particular bitrates level of compression. There are some albums that I have had to go back and retrieve a new copy of simply because the album demanded more dynamic headroom than what the lower bit rate provided.

A great example of this is Mastodon's Crack the Skye. I have had a copy of this album @ 196kbps for a while, and it has always kinda bothered me that at hard transient attacks the music seemed to get "squashed" simply because the bit rate was too small for such a dynamically demanding recording. I went and retrieved a copy of the album at 320kbps and before I deleted the old file I decided to put it to the test. I told my roommate to play the two different copies of the same song, to play them randomly and to not tell me which one she was playing first. THe difference was obvious. The 320 example sounded fuller, like it had room to breath because that extra headroom was available during transient attacks. There was a noticeable lack of distortion and overt sense of heavy compression at musical peaks, which is exactly what I experienced with the original smaller file. The soundstage sounded more 3-dimensional and lifelike , and the signal to noise ratio also seemed dramatically improved.

Guybrush

Of course I couldn't let people download mp3s to compare them so I had to come up with another way. The flash player, despite limitations, was the best solution I could think of.

Satch, I think you have to remember that I'm interested in all sorts of results. If enough people answer, I could look at how many manage to separate the lowest BR file from the highest for example. I can also see whether or not people are able to separate smaller differences in bitrate at different overall qualities. By having several files rather than just two for example, the chance of getting it right by luck becomes smaller and that should also be a strength, not a weakness.

I did think this through and I think the results will be interesting .. if I ever get some. I think people are either not interested or they're worried about getting it wrong.

I'm aware different genres and so on will differ in how easy they are to separate. That's why this is round 1, I thought I'd split it up a bit.


I'll post the correct answers sometime when I have enough data!

__________________
Something Completely Different

CAPTAIN CAVEMAN

i did the test but didn't send a private message because i couldn't tell any of them apart.

so... there you go.

Guybrush

I think the getting the sequence correct - yes or no - is only part of what's interesting. Of course someone could have 3 songs correct, but then shift the sequence by 4th song first when it should've been further up. In that case, the person would be wrong about 1 song, but correct about the BR relationship between the others.

The sequence would still be wrong so that sort of correctness is of course not shown in my little summary up there (yet). I might have to fix that.

__________________
Something Completely Different