Endless Paradigm

Welcome to my (hopefully) monthly review of the most popular Internet Browsers available, deciding what really is the BEST Browser available to the common computer user.

Part 1 & 2 will be focusing on Browser Performance and Part 3 will be focusing on Browser Design & Functionality.

In Version 1, the Wars will feature the following contenders:

Mozilla Firefox 3.0.6
Microsoft Internet Explorer 7
Google Chrome 1.0.154.48
Opera 9.63
Apple Safari 3.2.1
Maxthon 2.5.1

Also contending is their in-development counterparts:

Mozilla Firefox (Minefield) 3.2a1pre (Alpha 1 Preview)
(Shiretoko might be added in v1.1)
Mozilla SeaMonkey 2.0a3pre (Alpha 3 Preview)
Microsoft Internet Explorer 8 RC1 (Release Candidate 1)
Opera 10 Alpha 1
Google Chromium 2.0.160.1


These were the latest available on 9th February 2009.

---

I will be focusing on Browser Performance and measuring it based on 2 methods - Benchmarking & Real-Time.

Also, I will show the RAM usage on Startup, as well as after the Benchmarking and Real-Time tests, showing the difference in incrementation.

Benchmarking

I will use some of the most vigorous Browser tests available to measure their speed including: SunSpider, the Javascript Benchmark by WebKit, BenchJS, a second well-known Javascript Benchmark, a CSS Rendering test and of course, the Acid2 & Acid3 Compliance tests.

The Results each Browser gets will earn it marks out of 5 to simplify the view on the Browser performance against others, how exactly will be explained later.


The Test Machine is an Intel Pentium Dual-Core T2080 1.73GHz, with 443MB of Physical RAM, running Microsoft Windows Vista Home Basic.

---

~ Contents ~

a) Benchmarking
  a1) Acid2 & Acid3 Compliance
  a2) SunSpider JavaScript Benchmark
  a3) BenchJS
  a4) CSS Rendering

b) RAM Usage

c) Conclusion & Comment
d) Links

---

a1) Acid2 & Acid3 Compliance

Kicking off this Browser Performance is are the Acid Tests.

The Acid Tests are tests created by the Web Standards Project to allow Browser Developers to test to see if their Browser is compliant with the latest web standards out there; allowing Web Designers to breathe a little easier when it comes to what is supported and what is not.

For more information on the Tests or on how they work exactly, visit the Acid Tests site here: http://www.acidtests.org/


Okay, lets start of with Acid2!



Acid2

I will show Graphical Results so it makes it that much more interesting.

[Click Here for Reference Rendering]


Mozilla Firefox 3.0.6 - Pass

Microsoft Internet Explorer 7 - Fail

Google Chrome 1.0.154.48 - Pass

Opera 9.63 - Pass

Apple Safari 3.2.1 - Pass

Maxthon 2.5.1 - Fail

Mozilla Firefox 3.2a1pre - Pass

Mozilla SeaMonkey 2.0a3pre - Pass

Microsoft Internet Explorer 8 RC1 - Pass

Opera 10 Alpha 1 - Pass

Google Chromium 2.0.160.1 - Pass


No surprises there, the fresher engines can tackle this one (WebKit (Safari, Google Chrome & Chromium), Gecko 1.9.x (Firefox & SeaMonkey), Presto 2.x (Opera), Trident 8 (IE8)), but sadly, one of the most widespread browsers of the moment, Internet Explorer 7, which uses Trident 7, failed along with the Trident 7-Powered counterpart from China, Maxthon 2.5.1.


Acid3

[Click Here for Reference Rendering]


Mozilla Firefox 3.0.6 - 71/100 (Fail)

Microsoft Internet Explorer 7 - 12/100 (Fail)

Google Chrome 1.0.154.48 - 78/100 (Fail)

Opera 9.63 - 85/100 (Fail)

Apple Safari 3.2.1 - 74/100 (Fail)

Maxthon 2.5.1 - 14/100 (Fail)

Mozilla Firefox 3.2a1pre - 93/100 (Fail)

Mozilla SeaMonkey 2.0a3pre - 93/100 (Fail)

Microsoft Internet Explorer 8 RC1 - 20/100 (Fail)

Opera 10 Alpha 1 - 100/100 (Pass)

Google Chromium 2.0.160.1 - 100/100 (Pass, Linktest Failed[1])
[1] http://code.google.com/p/chromium/issues/detail?id=6208


Google Chromium has major success here, which shows that later builds of WebKit are fully compliant - which should mean when Safari 4 is finally released (it is currently at a Developmental Build) it will be too.

Mozilla have made very good progress from Gecko 1.9 (Firefox 3.0.x) to Gecko 1.9.1/1.9.2 (SeaMonkey 2.0a3pre & Firefox 3.2a1pre), and I would expect that once they are at Stable developmental stage (SeaMonkey & Firefox 3.1 (Shiretoko) coming first, which will both use Gecko 1.9.1), they will pass Acid3 with full marks.

Who knows if Microsoft are planning to make Internet Explorer 8 Compliant with Acid3.. but what wee do know is that at the moment it's far from the case and they really need to work on it to keep up with the competition.

It's nice to see Opera are keeping up to par as well with Opera 10 Alpha which uses Presto 2.2 (9.63 uses Presto 2.1.1).


a2) SunSpider JavaScript Benchmark

Now then, the SunSpider JavaScript Benchmark is a full on, multiple run (5 times), statistically accurate Benchmark from some of the people at WebKit.

For more information, check out the SunSpider JavaScript Benchmark here: http://www2.webkit.org/perf/sunspider-0....pider.html


There was tons of testing done and too much data to put here so I'll give an overview of the results.

Here are the Totals:

----------------------------------
Top Rank (<5000ms)
----------------------------------

Google Chromium 2.0.160.1 (9376)
2811.8ms ± 2.5%

Mozilla Firefox 3.2a1pre
3571.4ms ± 1.0%

Google Chrome 1.0.154.48
3853.6ms ± 4.6%

Mozilla SeaMonkey 2.0a3pre
4005.4ms ± 0.8%

----------------------------------
Mid Rank (<15000ms)
----------------------------------

Mozilla Firefox 3.0.6
9842.0ms ± 1.0%

Opera 10.0 Alpha 1 (1139)
11377.6ms ± 1.0%

Apple Safari 3.2.1
11452.4ms ± 0.4%

Opera 9.6.3
12812.8ms ± 0.6%

----------------------------------
Low Rank (15000>ms)
----------------------------------

Maxthon 2.5.1
17217.4ms ± 1.8%

Internet Explorer 8 RC1
17626.6ms ± 1.5%

Internet Explorer 7
89885.8ms ± 3.0%

----------------------------------

Well there you have it! Google Chromium is the fastest of the lot thanks to its V8 Engine, with the early Firefox 3.2a1pre not far behind and the Public Stable release of Google Chrome after that.

The current Firefox Stable leads the Mid Rankers which also have the likes of Apple Safari and Opera.

Opera 10 seems to be focusing more on Compliance rather than speed. After all, they are working on Carakan for Opera 11 (I'll talk some more of that later).

It's pretty clear that the next milestone for Browsers is a massive one. The speed of the Browsers using newer JavaScript engines are superb... with the exception of Internet Explorer 8 RC1.

I mean COME ON.. even Maxthon beat Internet Explorer 8 RC1. It's only plain to see how awful the Internet Explorer 7 JavaScript engine was in comparison, but it's been around for a few years so it's expected - but the fact that there hasn't been a major release since v7, wee're still waiting for it and lets hope it's relatively better for when its shipped with Windows 7. Come on Microsoft!!

You'll want to see the Datasheet if you want to see the stats in more detail.



a3) BenchJS

Sadly, the home of BenchJS has gone offline, but I found a mirror of it at a site apparently under construction... http://flaneurs.net/testFun/

Now then, BenchJS consists of 7 JavaScript Tests and while SunSpider DID test JavaScript earlier, more tests can't hurt, can they?

(Fastest are higher)

Test 1

Test 1 counts to a 10000 and displays the figures inside the status bar. At the same time it displays a red progress bar.

----------------------------------
            Quickest
Google Chrome 1.0.154.48
0.466secs

----------------------------------

Google Chromium 2.0.160.1 (9376)
0.557secs

Opera 10.0 Alpha 1 (1139)
0.937secs

Opera 9.6.3
1.14secs

Apple Safari 3.2.1
1.2secs

Mozilla Firefox 3.2a1pre
1.406secs

Mozilla Firefox 3.0.6
1.453secs

Internet Explorer 8 RC1
1.578secs

Internet Explorer 7
1.593secs

Maxthon 2.5.1
1.609secs

----------------------------------

Mozilla SeaMonkey 2.0a3pre
3.072secs
            Slowest
----------------------------------

Test 2

This page tests the popup performance of your PC, Browser and PC.

* Google Chrome/Chromium didn't have an option to enable the popups

----------------------------------
            Quickest
Opera 9.6.3
0.422secs

----------------------------------

Opera 10.0 Alpha 1 (1139)
0.691secs

Apple Safari 3.2.1
0.908secs

Mozilla Firefox 3.2a1pre
0.932secs

Mozilla Firefox 3.0.6
0.982secs

Maxthon 2.5.1
2.359secs

Internet Explorer 7
2.938secs

Google Chrome 1.0.154.48
N/A*

Google Chromium 2.0.160.1 (9376)
N/A*

----------------------------------

Internet Explorer 8 RC1
7.032secs
            Slowest
----------------------------------


Test 3

Test 3 replaces 300 tiny images as fast as possible. It repeats this procedure 10 times.

----------------------------------
            Quickest
Google Chromium 2.0.160.1 (9376)
0.381secs

----------------------------------

Opera 10.0 Alpha 1 (1139)
0.66secs

Google Chrome 1.0.154.48
0.679secs

Opera 9.63
0.703secs

Apple Safari 3.2.1
0.765secs

Mozilla Firefox 3.2a1pre
0.888secs

Mozilla Firefox 3.0.6
0.981secs

Internet Explorer 7
1.046secs

Mozilla SeaMonkey 2.0a3pre
1.157secs

----------------------------------

Internet Explorer 8 RC1
1.64secs

Maxthon 2.5.1
1.64secs
            Slowest
----------------------------------

Test 4

Test 4 manipulates a long text in different ways.

----------------------------------
            Quickest
Google Chromium 2.0.160.1 (9376)
0.123secs

----------------------------------

Google Chrome 1.0.154.48
0.144secs

Mozilla Firefox 3.0.6
0.325secs

Opera 10.0 Alpha 1 (1139)
0.354secs

Internet Explorer 8 RC1
0.359secs

Maxthon 2.5.1
0.359secs

Opera 9.63
0.36secs

Apple Safari 3.2.1
0.373secs

Mozilla Firefox 3.2a1pre
0.626secs

Mozilla SeaMonkey 2.0a3pre
0.846secs


----------------------------------

Internet Explorer 7
1.969secs
            Slowest
----------------------------------


Test 5

Test 5 creates 2000 table-cells and calculates a random backgroundcolor for each table-cell.

----------------------------------
            Quickest
Opera 10.0 Alpha 1 (1139)
0.09secs

----------------------------------

Opera 9.63
0.109secs

Google Chrome 1.0.154.48
0.185secs

Google Chromium 2.0.160.1 (9376)
0.185secs

Internet Explorer 8 RC1
0.188secs

Apple Safari 3.2.1
0.205secs

Mozilla Firefox 3.0.6
0.21secs

Mozilla SeaMonkey 2.0a3pre
0.312secs

Maxthon 2.5.1
0.359secs

Mozilla Firefox 3.2a1pre
0.626secs

----------------------------------

Internet Explorer 7
1.109secs
            Slowest
----------------------------------


Test 6

Test 6 creates a phrase out of 50 different layers that are pulled together.

----------------------------------
            Quickest
Opera 10.0 Alpha 1 (1139)
1.889secs

----------------------------------

Opera 9.63
2secs

Google Chrome 1.0.154.48
2.059secs

Google Chromium 2.0.160.1 (9376)
2.075secs

Mozilla Firefox 3.2a1pre
2.54secs

Mozilla Firefox 3.0.6
3.289secs

Mozilla SeaMonkey 2.0a3pre
3.372secs

Maxthon 2.5.1
3secs

Internet Explorer 7
3.843secs

Internet Explorer 8 RC1
3.984secs

----------------------------------

Apple Safari 3.2.1
6.888secs
            Slowest
----------------------------------


Test 7

Test 7 calculates the days of the week for the next 10000 x-mas. IMPORTANT: The test considers only the calculating-time NOT the time required to display the result!

----------------------------------
            Quickest
Mozilla Firefox 3.2a1pre
0.231secs

----------------------------------

Mozilla Firefox 3.0.6
0.25secs

Apple Safari 3.2.1
0.253secs

Mozilla SeaMonkey 2.0a3pre
0.26secs

Google Chromium 2.0.160.1 (9376)
0.293secs

Google Chrome 1.0.154.48
0.32secs

Opera 9.63
0.375secs

Opera 10.0 Alpha 1 (1139)
0.379secs

Maxthon 2.5.1
0.469secs

Internet Explorer 8 RC1
0.469secs

----------------------------------

Internet Explorer 7
1.5secs
            Slowest
----------------------------------

That's all of them, as expected Google Chrome & Chromium were both consistant, not always the quickest but always in the top half.

Internet Explorer 7 as well as 8 RC1 choked on quite a few of the tests but Maxthon proved to be more efficient in their own catagory.

Opera was consistant, neither 9.63 or 10 Alpha fell to the bottom 3 and 10 Alpha topped 2 of the tests with 9.63 just behind.

Fairly average performance from Mozilla's Browsers in 5/6 of the tests, with SeaMonkey usually performing worse, although Firefox 3.2a1pre topped Test 7, which makes sense since it beat every other Browser in the SunSpider Math Tests (see Datasheet).

Apple Safari was very mixed, except in Test 6 where it ran the test very poorly with terrible consistancy during processing, otherwise nothing too disappointing.

I do consider these tests important, as it deals with more hands-on processes and uses of Javascript.



a4) CSS Rendering

CSS (Cascading Style-shit) Rendering times are important to make up the sites structure and a vast amount of sites use CSS, so, the quicker is always the better when it comes to making things look nice.

This is a single table in order of the quickest.

Something to keep in mind is that the 1st load is a full load, whereas the 2nd and 3rd loads are loaded from cache.

----------------------------------
            Quickest
Apple Safari 3.2.1

1st Load: 183ms
2nd Load: 190ms
3rd Load: 186ms

----------------------------------

Google Chrome 1.0.154.48

1st Load: 287ms
2nd Load: 257ms
3rd Load: 260ms

Google Chromium 2.0.160.1 (9376)

1st Load: 297ms
2nd Load: 276ms
3rd Load: 271ms

Opera 10.0 Alpha 1 (1139)

1st Load: 418ms
2nd Load: 326ms
3rd Load: 475ms

Opera 9.63

1st Load: 531ms
2nd Load: 421ms
3rd Load: 453ms

Maxthon 2.5.1

1st Load: 828ms
2nd Load: 859ms
3rd Load: 938ms

Internet Explorer 7

1st Load: 1265ms
2nd Load: 1125ms
3rd Load: 1172ms

Mozilla Firefox 3.2a1pre

1st Load: 1123ms
2nd Load: 1275ms
3rd Load: 1211ms

Internet Explorer 8 RC1

1st Load: 1516ms
2nd Load: 1250ms
3rd Load: 1406ms

Mozilla SeaMonkey 2.0a3pre

1st Load: 1383ms
2nd Load: 1561ms
3rd Load: 1487ms

----------------------------------

Mozilla Firefox 3.0.6

1st Load: 1805ms
2nd Load: 1547ms
3rd Load: 1441ms
            Slowest
----------------------------------


Extremely disappointing to see Firefox 3.06 be rock bottom in this test, none of the other Mozilla Browsers did much better.

Safari strut its stuff here with a fantastic result... Apple still have that styling edge (eheh).

Google Chrome & Chromium weren't far behind though, showing that WebKit really is the quickest Layout Engine in this case.

Presto was excellent for Opera 9.63 & 10 Alpha being the next two in the results.

Maxthon clearly showed off that it works more efficiently compared to Internet Explorer 7 or 8 RC1.


b) RAM Usage

I decided to keep record of the RAM Usage while I tested the Browsers, as it's still considered quite an important factor.

I'd like to remind you that I DISABLED Plugins/Addons before testing as well.


(in no particular order)
----------------------------------

Mozilla Firefox 3.0.6
Startup: 16~MB
After Tests: 46~MB
Increment: 30~MB

Mozilla Firefox 3.2a1pre
Startup: 19~MB
After Tests: 59~MB
Increment: 40~MB

Opera 9.63
Startup: 24~MB
After Tests: 34~MB
Increment: 10~MB

Opera 10 Alpha
Startup: 21~MB
After Tests: 31~MB
Increment: 10~MB

Google Chrome (Plugins cannot be disabled within Options)
Startup: 6~MB + 7~MB
After Tests: 16~MB + 16~MB
Increment: 10~MB + 9~MB

Google Chromium (Plugins cannot be disabled within Options)
Startup: 6~MB + 6~MB
After Tests: 11~MB + 15~MB
Increment: 5~MB + 9~MB

Microsoft Internet Explorer 7
Startup: 6~MB
After Tests: 16~MB
Increment: 10~MB

Microsoft Internet Explorer 8 RC1
Startup: 4MB + 4MB
After Tests: 6~MB + 31~MB
Increment: 2~MB + 27~MB

Apple Safari 3.2.1
Startup: 49~MB
After Tests: 56~MB
Increment: 7~MB

SeaMonkey 2.0a3pre
Startup: 20~MB
After Tests: 38~MB
Increment: 18~MB

Maxthon 2.5.1
Startup: 12~MB
After Tests: 23~MB
Increment: 11~MB

----------------------------------

Apple Safari uses a massive 49~MB at startup, so make sure you avoid this browser unless you do have RAM to spare; keep in mind however, it had the smallest increment of only 7MB after the tests.

Internet Explorer 7 used the least amount of RAM, but it doesn't compensate for its lack of functionality at all.

A rather large 29MB increment from Internet Explorer 8 RC1, but an even more rediculous 30~MB & 40~MB increment from the Firefox Browsers.

Both Opera Browsers, Google Chrome & Chromium and Maxthon did very well and SeaMonkey wasn't too bad.


d) Links

Senseito's Browser Test Feb 2009.pdf - http://www.sendspace.com/file/i4v3pr

[1] http://code.google.com/p/chromium/issues/detail?id=6208

SunSpider - http://www2.webkit.org/perf/sunspider-0....pider.html
BenchJS - http://flaneurs.net/testFun/
CSS Rendering Test - http://www.howtocreate.co.uk/csstest.html



c) Conclusion & Comment


SO... ultimately, the obvious fact is, Google's V8 is going to rule JavaScript processing for a long while... Chromium is undeniably impressive.

While Mozilla's work is still nevertheless fantastic, there is still some catchup which can probably be expected in Stable versions of Firefox 3.1 & 3.2, hopefully.

Opera was in no way BAD as such, bit not that impressive, either. Opera 10 Alpha is in early stages and I am looking forward to seeing how it improves over time.

Also, I mentioned Carakan too, for more info, check this out: http://my.opera.com/core/blog/2009/02/04/carakan

Opera Core Concerns Wrote:So how fast is Carakan? Using a regular cross-platform switch dispatch mechanism (without any generated native code) Carakan is currently about two and a half times faster at the SunSpider benchmark than the ECMAScript engine in Presto 2.2 (Opera 10 Alpha).

If that's true, that'll bring Opera back in the game, BUT of course... wee'll be waiting for Opera 11, who knows when wee'll expect that... by the time it gets released its competitors could very well be even better than they are now, making Opera just one milestone slower than the rest (except for IE, of course).

The numbers for Internet Explorer 7 & 8 are plain to see, really, and honestly the future for the browser remains doubtful to me, solely relying on its intergration into Windows.

Maxthon is quite the impressive thing, and might be a better alternative to IE for those who like IEs engine, but honestly, there isn't much point.

I'm not gonna say get so on or so forth, that'll come at the end of Part 3.

If you took your time to check this out, thanks for reading and I hope you're looking forward to the next couple of parts.

Till next time!

Senseito Sakura

Nice written review Sensei! I only did a quick overview because I'm tired but you obviously spend alot of time and work into that! I think Zinga should pin it because it's a great reference for everyone who is undecided about his browser.

Nice review!! It seems that Mozilla isn't doing so well on performance :(
I admit that youtube doesn't seem to work properly for me in FF so I use IE -_-

MehHakker Wrote:Nice review!! It seems that Mozilla isn't doing so well on performance :(
I admit that youtube doesn't seem to work properly for me in FF so I use IE -_-

So it isn't just me then... I've had one or two mishaps with Flash on Firefox as well...

God save ure soul from using IE... HAHA

mmm...
I question some of the results...

I also question how the referance image works because I.E. and firefox have two completely different rendering engines...

also some web browser may be designed to be more efficient with more power...
like maybe certain web browser #1 runs better with lots of ram instead of certain web browser #2
where as certain web browser #2 runs on lower ram while web browser #1 needs more to be more efficient...

still must have been a hell of a lot a work and I appreciate you even trying to investigate it but somewhere in my mind I feel there are far too many variables for the results to be accurate....
also I feel it probably more solely lies on what the user wants/loves
and since people are so different it is usually hard for another person to tell them what is best for them
because sometimes the only person who really knows, is you
even though

---

also running acid 3 on firefox on my machine (better specs than that one) passes

Senseito URΩBΩROS Wrote:

MehHakker Wrote:Nice review!! It seems that Mozilla isn't doing so well on performance :(
I admit that youtube doesn't seem to work properly for me in FF so I use IE -_-

So it isn't just me then... I've had one or two mishaps with Flash on Firefox as well...

God save ure soul from using IE... HAHA

Please forgive me Father for I have sinned!

Wow, that's quite a review.  A lot of stats :P I'd suggest putting them into a graph :) (I can make one for you if you want)
IE's lack of RAM usage would be because it's rendering engine is a part of Windows (can be stripped, but it'll cause IE not to work at all), thus loading the actual browser only really loads the interface.  A bit unfair, but not much you can do about it :P

@Slushba132: very true, but this is a failing in most synthetic benchmarks - ie like how PCMark measures your system performance.

ZiNgA BuRgA Wrote:Wow, that's quite a review.  A lot of stats :P I'd suggest putting them into a graph :) (I can make one for you if you want)

Please. I wanted to but I only had Google Spreadsheets to work with. Which was great, though, helped me make the PDF Datasheet.

ZiNgA BuRgA Wrote:IE's lack of RAM usage would be because it's rendering engine is a part of Windows (can be stripped, but it'll cause IE not to work at all), thus loading the actual browser only really loads the interface.  A bit unfair, but not much you can do about it :P

Thought so. Thanks for clarifying on that.

Slushba132 Wrote:mmm...
I question some of the results...

I also question how the referance image works because I.E. and firefox have two completely different rendering engines...

Nevertheless it should be a smooth animation (the time taken is measured in test) and the completed image SHOULD appear to the almost perfect specification shown... - All the Acid2 Passes were the same except for Apple Safari 3.2.1 which uses more AntiAlias on the fonts. A Full pass is a full pass (completing the test successfully makes the browser compliant).

Slushba132 Wrote:also some web browser may be designed to be more efficient with more power...
like maybe certain web browser #1 runs better with lots of ram instead of certain web browser #2
where as certain web browser #2 runs on lower ram while web browser #1 needs more to be more efficient...

While that might be true, the end user will download software based on its claims; a software that claims it will be faster with more RAM will have the user know that fact - despite that fact, so long as that Physical RAM is available it shall be used (Apple Safari, for example).

Slushba132 Wrote:still must have been a hell of a lot a work and I appreciate you even trying to investigate it but somewhere in my mind I feel there are far too many variables for the results to be accurate....

That is why I like the SunSpider Benchmark - it runs all the tests 5 times over and gives percentage variability in the results.

Slushba132 Wrote:also I feel it probably more solely lies on what the user wants/loves
and since people are so different it is usually hard for another person to tell them what is best for them
because sometimes the only person who really knows, is you

So true, but hey, nobody knew how good Firefox 2 was until they installed it.. same goes for when Google released Chrome.

I will divulge into these arguments and needs in Part 3.

---

Slushba132 Wrote:also running acid 3 on firefox on my machine (better specs than that one) passes

Can you tell me what version of Firefox you are running? And a picture off the big 100/100 because its known not to pass Acid3 ;)

Computer specifications are only relevant to the Javascript and CSS Benchmarks - Acid2/3 is designed to test a Browsers Standards Compliance.

ha ha never mind.. I'm a dumb donkey
looking at the reference lol

Slushba132 Wrote:ha ha never mind.. I'm a dumb donkey
looking at the reference lol