Update: Firefox 3.5 is fast, but still behind Chrome and Safari

Chicago (IL) – About two weeks ago, following the release of Safari 4.0, we had a close look at the performance delivered by today’s browsers. At the time, our benchmark runs were based on the first release candidate build of Firefox 3.5, which was well behind the WebKit engine-based Chrome and Safari browsers. Now that we have the final version, we wondered: Did Firefox gain more speed?

The opinions about Apple’s Safari browser may be divided and you may prefer the interface and features of other browsers, but it simply cannot be denied that Safari is, in terms of performance, your best bet these days. According to our benchmark from earlier this month, Safari beats the performance leader Chrome in the Peacekeeper, Celtic Kane and Slick Speed benchmark suites, while being a close second in Google’s own V8 and the WebKit SunSpider tests.

Firefox 3.5 RC1 trailed the two browsers in every JavaScript benchmark and has only an edge in Flash performance. So how about 3.5 final?

We were not able to complete the Acid 3 web standard compatibility test, due to an obvious high volume on the website. The only (incomplete) run that yielded a number came in at 68 of 100, which we believe to be incorrect. At this time, there is no reason to believe that the performance has decreased from version RC1. Our previous from June 15 with Firefox 3.5 RC1:

1. Safari: 100/100
2. Chrome: 100/100, Linktest failed
3. Firefox: 93/100
4. IE: 20/100

(Higher numbers are better)

Next up was Google’s Chrome V8 benchmark. Firefox scored 355 points which is on one level with the June 15 result of 356 points:

1. Chrome: 2709
2. Safari: 2503
3. Firefox: 355 (356 on June 15)
4. IE: 81

(Higher numbers are better)

SunSpider, which tests the core JavaScript language performance only, revealed a similar picture. Firefox 3 came in at 1204.3 mseconds, which is slightly behind the June 15 performance, but still within the margin of error.

1. Chrome: 628.4
2. Safari: 683.8
3. Firefox: 1204.3 (1162.8 on June 15)
4. IE: 5069.4

(Lower numbers are better)


In Celtic Kane, another JavaScript benchmark, Firefox 3.5 final, was also a bit slower than the RC1 version (results given in mseconds).

1. Safari: 84
2. Chrome: 183
3. Firefox: 254 (233 on June 30)
4. IE: 512    

(Lower numbers are better)

Slick Speed rounds out the JavaScript benchmark run, showing a performance that is on par with RC1. However, it is interesting to see that this is the only JavaScript test in which the performance of Firefox has slightly increased.

1. Safari: 38, 31, 149, 124, 1
2. Chrome: 42, 23, 209, 144, 44
3. Firefox:  68, 69, 167, 176, 29 (71, 71, 187, 177, 38 on June 15)
4. IE: 205, 183, 739, 497, 193

(Lower numbers are better)

So, what about non-JavaScript benchmarks?

We took Firefox 3.5 through Futuremark’s Peacekeeper test, which yielded a tiny improvement from 1964 to 1989 points on our modest quad-core PC.

1. Safari: 3671
2. Firefox: 1989 (1964 on June 15)
3. IE: 804
4. Chrome: Not supported

(Higher numbers are better)

All those benchmarks show that the performance of the final version is virtually flat with RC1. However, we noticed one very dramatic improvement we did not expect: Flash performance.

Firefox already led the pack in Flash performance before, but it appears that the final version of 3.5 brings yet another substantial enhancement. Our test run of the Le Crabe test yielded a stunning 636 simulations of animated objects, 77% more than the next best browser, Microsoft’s Internet Explorer.

1. Firefox: 636 (356 on June 15)
2. IE: 356
2. Safari: 256
3. Chrome: 241    

(Higher numbers are better)

Other than the Flash performance, it is clear that there are no new performance improvements in Firefox 3.5 over version RC1. At least if we believe those benchmarks, it appears that Firefox is well behind Chrome and Safari, while real-life browsing may actually not reveal the performance differences between those browsers. Opera’s web browser was not part of this test as its market share is below 1%, according to Net Applications. However, we will follow up with a comprehensive benchmark article soon, which will include Opera.

You can run these benchmarks yourself to see how the browsers perform on your computer. Just keep in mind that the yielded scores depend on the horsepower of your computer hardware and it is unlikely that you will see numbers similar to those published in this article. Our test system was an Intel Core 2 Quad Q6600-based system with 4 GB of memory.  

The software versions were Safari 4.0.530.17, Internet Explorer 8.0.6001.18783, Chrome and Firefox 3.5 build 3.5.30729.