Feeds

Google, Microsoft butt heads in browser benchmark battle

Will the real real-world test please stand up?

Secure remote control for conventional and virtual desktops

Google Chrome has finally met a web benchmark suite it can't master, and no one could be happier than the Internet Explorer team at Microsoft, which has used the occasion to suggest that the Chocolate Factory's browser is not really the speed demon it's cracked up to be.

Dubbed RoboHornet, the new, open source benchmark suite is different from earlier ones in that it doesn't focus on timing arbitrary JavaScript algorithms. Instead, it collects code snippets found in popular web frameworks and applications – including jQuery, YUI, Google Apps, and others – in an attempt to demonstrate real-world performance problems.

The result is a formidable benchmark suite that takes minutes to complete on typical hardware – assuming the browser can make it through the full list of tests at all.

"It's a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points," says Google Chrome product manager Alex Komoroske, who launched the project.

That a Google staffer came up with the idea for RoboHornet should surprise no one, since the ad-slinging giant has been responsible for several earlier browser benchmarks. What is surprising, on the other hand, is that Chrome doesn't always earn the highest score, depending on which you run it on.

In tests conducted by Tom's Hardware, Internet Explorer 10 scored as much as 22 per cent higher than the most recent stable version of Chrome when running on Windows 8, and Safari bested Chrome by a similar margin on OS X.

Those findings did not go unnoticed by Microsoft, but despite the strong showing by its latest browser, Redmond was not impressed – not with IE's benchmark scores and not with RoboHornet itself.

"While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn't all that representative of the performance users might encounter on real-world sites," wrote Microsoft's Roger Capriotti in a blog post on Tuesday.

To prove it, Microsoft engineers rolled their own version of RoboHornet that they're calling RoboHornet Pro, which Capriotti claims does a better job of modeling what goes on inside the browser when a real-world web app is running.

Redmond's version of the benchmark runs a Matrix-like animation in the foreground while its tests are running in the background. Although Microsoft's truncated test suite takes a lot less time to complete, the animation stutters to a halt on Chrome after a few seconds, while IE10 keeps both the animation and the benchmarks running in tandem.

According to Capriotti, that's because RoboHornet is merely the latest in a long line of browser "micro-benchmarks", which really only demonstrate isolated aspects of browser performance under lab-like conditions.

"While we will look to see how we stack up on other browser vendors' tests, our focus will remain on real-world site performance," he wrote.

Neither Komoroske or any of the other RoboHornet code stewards has commented directly on RoboHornet Pro or Microsoft's findings, but Komoroske at least acknowledges that there is more work to be done.

"RoboHornet is still in a very early alpha state, but that's where you come in," Komoroske writes. "It's up to you and to propose and vote for performance issues you care about, helping shape the future of the benchmark and effectively defining the areas that browser vendors will invest in making run faster!"

Developers who want to get involved can participate via the project's GitHub page. ®

Security for virtualized datacentres

More from The Register

next story
UNIX greybeards threaten Debian fork over systemd plan
'Veteran Unix Admins' fear desktop emphasis is betraying open source
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Redmond top man Satya Nadella: 'Microsoft LOVES Linux'
Open-source 'love' fairly runneth over at cloud event
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
Admins! Never mind POODLE, there're NEW OpenSSL bugs to splat
Four new patches for open-source crypto libraries
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.