Feeds

Google, Microsoft butt heads in browser benchmark battle

Will the real real-world test please stand up?

Internet Security Threat Report 2014

Google Chrome has finally met a web benchmark suite it can't master, and no one could be happier than the Internet Explorer team at Microsoft, which has used the occasion to suggest that the Chocolate Factory's browser is not really the speed demon it's cracked up to be.

Dubbed RoboHornet, the new, open source benchmark suite is different from earlier ones in that it doesn't focus on timing arbitrary JavaScript algorithms. Instead, it collects code snippets found in popular web frameworks and applications – including jQuery, YUI, Google Apps, and others – in an attempt to demonstrate real-world performance problems.

The result is a formidable benchmark suite that takes minutes to complete on typical hardware – assuming the browser can make it through the full list of tests at all.

"It's a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points," says Google Chrome product manager Alex Komoroske, who launched the project.

That a Google staffer came up with the idea for RoboHornet should surprise no one, since the ad-slinging giant has been responsible for several earlier browser benchmarks. What is surprising, on the other hand, is that Chrome doesn't always earn the highest score, depending on which you run it on.

In tests conducted by Tom's Hardware, Internet Explorer 10 scored as much as 22 per cent higher than the most recent stable version of Chrome when running on Windows 8, and Safari bested Chrome by a similar margin on OS X.

Those findings did not go unnoticed by Microsoft, but despite the strong showing by its latest browser, Redmond was not impressed – not with IE's benchmark scores and not with RoboHornet itself.

"While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn't all that representative of the performance users might encounter on real-world sites," wrote Microsoft's Roger Capriotti in a blog post on Tuesday.

To prove it, Microsoft engineers rolled their own version of RoboHornet that they're calling RoboHornet Pro, which Capriotti claims does a better job of modeling what goes on inside the browser when a real-world web app is running.

Redmond's version of the benchmark runs a Matrix-like animation in the foreground while its tests are running in the background. Although Microsoft's truncated test suite takes a lot less time to complete, the animation stutters to a halt on Chrome after a few seconds, while IE10 keeps both the animation and the benchmarks running in tandem.

According to Capriotti, that's because RoboHornet is merely the latest in a long line of browser "micro-benchmarks", which really only demonstrate isolated aspects of browser performance under lab-like conditions.

"While we will look to see how we stack up on other browser vendors' tests, our focus will remain on real-world site performance," he wrote.

Neither Komoroske or any of the other RoboHornet code stewards has commented directly on RoboHornet Pro or Microsoft's findings, but Komoroske at least acknowledges that there is more work to be done.

"RoboHornet is still in a very early alpha state, but that's where you come in," Komoroske writes. "It's up to you and to propose and vote for performance issues you care about, helping shape the future of the benchmark and effectively defining the areas that browser vendors will invest in making run faster!"

Developers who want to get involved can participate via the project's GitHub page. ®

Secure remote control for conventional and virtual desktops

More from The Register

next story
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Sway: Microsoft's new Office app doesn't have an Undo function
Content aggregation, meet the workplace ... oh
Sign off my IT project or I’ll PHONE your MUM
Honestly, it’s a piece of piss
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
NetWare sales revive in China thanks to that man Snowden
If it ain't Microsoft, it's in fashion behind the Great Firewall
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.