Endless Paradigm

I decided to test various audio codecs to find out what are ideal quality settings to use.  Note, as said above, this isn't accurate.  At times, I've even had discrepancies with my own results :S

Just a few notes though:

• I've only tested using one song, which happens to be a 192kbps MP3.  This is definitely not a good choice of source, but is fine for my purposes - that is to test codecs at low bitrates
  
• I'm using the PSP headphones (not a good set of headphones to test either)
  
• When the encoder offered an option between speed and quality, the highest quality setting was always selected.
  
• The source file being an MP3 one, the results may favour MP3s...
  
• This is still somewhat incomplete

Constant Bitrate (CBR) Tests
(out of the two encoding methods, CBR is the most basic and provides the worst compression.  However, it's useful for comparison purposes (compatibility shouldn't be an issue) - ie providing a general idea of which codec is better)
Note that OGG Vorbis doesn't support CBR encoding, thus it's not included here.

Encoder/Settings	Bitrate (kbps)	Rating
Original MP3 file	192	100%
MPEG-1 Audio Layer 3 (MP3) (Note, JS = Joint Stereo)
LAME (JS)	64	30%
LAME (JS)	96	70%
SoundForge (JS)	112	65%
LAME (JS)	112	80%
SoundForge	112	60%
LAME	112	75%
SoundForge (JS)	128	90%
LAME (JS)	128	90%
SoundForge	128	90%
LAME	128	90%
WMP 10	128	90%
iTunes (JS)	128	82%
iTunes	128	71%
SoundForge (JS)	160	100%
LAME (JS)	160	100%
SoundForge	160	100%
LAME	160	100%
Windows Media Audio 9 Note: the "professional" codec has VERY limited support
WMA9 Std	96	55%
WMA9 Std	128	94%
WMA9 Pro	128	95%
Advanced Audio Coding, Low Complexity (LC-AAC) Note: I found Nero's encoder produced better quality than CT's.  But CT did produce better sounding bass than Nero's, but I felt that Nero produced overall better sound.
CodingTechnologies	64	60%
iTunes	64	64%
Nero	64	64%
CodingTechnologies	80	70%
iTunes	80	72%
Nero	80	72%
CodingTechnologies	96	83%
iTunes	96	85%
Nero	96	85%
CodingTechnologies	112	88%
iTunes	112	90%
Nero	112	92%
CodingTechnologies	128	96%
iTunes	128	97%
Nero	128	97%
Advanced Audio Coding, High Efficiency (HE-AAC)
Nero	24	60%
Nero	32	68%
Nero	48	88%
3GPP	48	85%
CodingTechnologies	48	86%
Nero	64	92%
CodingTechnologies	64	92%
Nero	80	94%
CodingTechnologies	80	94%
Nero	128	98%
CodingTechnologies	128	98%
HE-AAC with Parametric Stereo (HE-AAC v2) Note, PS seems to increase the quality of HE-AAC only at extremely low bitrates.  At higher bitrates, I found that it changed the sound way too much, and actually _reduced_ the quality
Nero	16	40%
Nero	24	65%
Nero	32	70%
Nero	48	72%
3GPP	48	85%
CodingTechnologies	48	74%
Nero	64	73%
Adaptive Transform Acoustic Coding (ATRAC3plus) I only bothered testing this codec actually because the PSP supports it
SoundForge	48	45%
SoundForge	64	82%
SoundForge	96	84%
SoundForge	128	88%
SoundForge	192	92%

Variable Bitrate (VBR) Tests
(VBR will typically produce better quality sounds than CBR with about the same filesize.  There are usually two methods, VBR (quantizer/quality) and ABR (average bitrate).  VBR will produce better results than ABR, so I've usually opted testing VBR (ABR gives better control over filesize))
Note, the average bitrate is a simple filesize ÷ time calculation.

Quality Setting	Av. Bitrate (kbps)	Rating
MPEG-1 Audio Layer 3 (MP3) (Note, JS = Joint Stereo)
SoundForge (JS)
15%	89	72%
20%	93	74%
50%	121	89%
75%	160	90%
LAME (JS)
Q9	89	75%
Q8	100	89%
Q7	109	90%
Q6	114	92%
Q5	121	95%
Q4	145	100%
Q2	184	100%
Windows Media Audio 9 Note: the "professional" codec has VERY limited support
WMA9 Standard
75%	108	88%
90%	169	98%
98%	270	100%
WMA9 Professional
75%	98	93%
90%	140	98%
98%	216	100%
OGG Vorbis (OggEnc2)
Q-1	43	71%
Q0	59	88%
Q1	77	99%
Q2	85	99%
Q4	126	100%
Advanced Audio Coding, Low Complexity (LC-AAC)
Nero
20%	87	86%
25%	106	90%
FAAC
"100" (out of 500)	128	96%
Advanced Audio Coding, High Efficiency (HE-AAC) (Nero)
7%	21	55%
15%	40	88%
20%	50	90%
25%	63	92%
50%	112	92%

Yeah, there's quite a few odd results in there, but oh well.


Sound Samples
Note that these are different from the ones I used for the above tests - these are mainly shorter:

Hope it's of some use :)

ZiNgA BuRgA Wrote:...I'll need to complete the table later...   For now, it should be okay enough for a guide.  Yeah, there's quite a few odd results in there, but oh well.

Hope it's of some use :)

thanks,

But it's been a while now, so are there still plans to complete this later?

I'm kinda interested in the results of vorbis and wma10 pro. ... oh, and why aren't there results for the lower ranges of HE-AAC+PS?

whenever i use VRB it sounds like 100% quality though.

Assassinator Wrote:thanks,

But it's been a while now, so are there still plans to complete this later?

I'm kinda interested in the results of vorbis and wma10 pro. ... oh, and why aren't there results for the lower ranges of HE-AAC+PS?

Urgh, forgot about this, I'll see if I can do it soon.
Can't encode into WMA10Pro...  WMP11 doesn't seem to encode into it.
Vorbis is a little below HE-AAC in terms of compression.

I'll consider uploading a few samples too.

Okay, some things added.  Sound samples up too.
Also, IDK why I rated WMA9 75% VBR so low...  oh well, that's kinda fixed. ;)

ZiNgA BuRgA Wrote:Note for HE-AAC and HE-AAC v2 samples, make sure you're using a SBR/PS capable player, such as Winamp.  Do not use players which do not decode SBR/PS such as iTunes/QuickTime.

I thought iTunes does playback HE-AAC, like how it would play back LC, thus effectively making the HE file sound like an LC file with half the bitrate = poo poo.

IMO being able to play it, but not properly is even worse than not being able to play it. I mean either play it propperly or don't play it at all, so to force people to use another player, which does play properly, so that way, people always gets the best out of the music.

Apple wants compatibility with the iPod.  iPods can't decode SBR (neither does the PSP)...  Meh.

^ Lawl, off-topicness, eh?
I stuffed it up trying to make a thing to allow users to change their colours.  And phpMyAdmin is being gay...