Feeds

Google, Microsoft butt heads in browser benchmark battle

Will the real real-world test please stand up?

Beginner's guide to SSL certificates

Google Chrome has finally met a web benchmark suite it can't master, and no one could be happier than the Internet Explorer team at Microsoft, which has used the occasion to suggest that the Chocolate Factory's browser is not really the speed demon it's cracked up to be.

Dubbed RoboHornet, the new, open source benchmark suite is different from earlier ones in that it doesn't focus on timing arbitrary JavaScript algorithms. Instead, it collects code snippets found in popular web frameworks and applications – including jQuery, YUI, Google Apps, and others – in an attempt to demonstrate real-world performance problems.

The result is a formidable benchmark suite that takes minutes to complete on typical hardware – assuming the browser can make it through the full list of tests at all.

"It's a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points," says Google Chrome product manager Alex Komoroske, who launched the project.

That a Google staffer came up with the idea for RoboHornet should surprise no one, since the ad-slinging giant has been responsible for several earlier browser benchmarks. What is surprising, on the other hand, is that Chrome doesn't always earn the highest score, depending on which you run it on.

In tests conducted by Tom's Hardware, Internet Explorer 10 scored as much as 22 per cent higher than the most recent stable version of Chrome when running on Windows 8, and Safari bested Chrome by a similar margin on OS X.

Those findings did not go unnoticed by Microsoft, but despite the strong showing by its latest browser, Redmond was not impressed – not with IE's benchmark scores and not with RoboHornet itself.

"While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn't all that representative of the performance users might encounter on real-world sites," wrote Microsoft's Roger Capriotti in a blog post on Tuesday.

To prove it, Microsoft engineers rolled their own version of RoboHornet that they're calling RoboHornet Pro, which Capriotti claims does a better job of modeling what goes on inside the browser when a real-world web app is running.

Redmond's version of the benchmark runs a Matrix-like animation in the foreground while its tests are running in the background. Although Microsoft's truncated test suite takes a lot less time to complete, the animation stutters to a halt on Chrome after a few seconds, while IE10 keeps both the animation and the benchmarks running in tandem.

According to Capriotti, that's because RoboHornet is merely the latest in a long line of browser "micro-benchmarks", which really only demonstrate isolated aspects of browser performance under lab-like conditions.

"While we will look to see how we stack up on other browser vendors' tests, our focus will remain on real-world site performance," he wrote.

Neither Komoroske or any of the other RoboHornet code stewards has commented directly on RoboHornet Pro or Microsoft's findings, but Komoroske at least acknowledges that there is more work to be done.

"RoboHornet is still in a very early alpha state, but that's where you come in," Komoroske writes. "It's up to you and to propose and vote for performance issues you care about, helping shape the future of the benchmark and effectively defining the areas that browser vendors will invest in making run faster!"

Developers who want to get involved can participate via the project's GitHub page. ®

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
You stupid BRICK! PCs running Avast AV can't handle Windows fixes
Fix issued, fingers pointed, forums in flames
Microsoft: Your Linux Docker containers are now OURS to command
New tool lets admins wrangle Linux apps from Windows
Facebook, working on Facebook at Work, works on Facebook. At Work
You don't want your cat or drunk pics at the office
Soz, web devs: Google snatches its Wallet off the table
Killing off web service in 3 months... but app-happy bonkers are fine
First in line to order a Nexus 6? AT&T has a BRICK for you
Black Screen of Death plagues early Google-mobe batch
prev story

Whitepapers

Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Designing and building an open ITOA architecture
Learn about a new IT data taxonomy defined by the four data sources of IT visibility: wire, machine, agent, and synthetic data sets.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?