Shocker: Chrome leads the the pack in new tests
The new suite includes all of the tests that were part of the Chocolate Factory's earlier V8 Benchmark Suite but adds a new set of benchmarks that were created from full versions of well-known web applications and libraries.
"A high score in the new benchmarks directly translates to better and smoother performance in similar web applications," wrote Chrome project manager Stefano Cazzulani in a post to the official Chromium blog on Tuesday.
The Octane suite gives an individual score for each test, plus a final score that is the geometric mean of all of the individual scores. Google says that it will keep adding new tests to the suite over time, so the final scores from one version of the suite will not be directly comparable to scores from other versions.
The individual tests will never change once released, however, which means individual test scores can be compared to results from different versions of the suite.
Just for kicks, your Reg correspondent took the new suite out for a spin with a few browsers. Not surprisingly, perhaps, Google's own app beat the competition, with Chrome 21 earning a final score 8653, while Firefox 14 only scored 5790 on the same hardware. The scores were based on an average of three runs.
Benchmarking Internet Explorer proved more problematic, however. We didn't have a Windows 8 machine with comparable specs handy, and it was no surprise that the IE 10 Preview struggled to complete the suite on a Samsung Slate fondleslab, scoring only 1661.
On the other hand, IE 9 wouldn't run the suite at all. In fact, it wouldn't even render its homepage properly. Are we surprised? ®