Feeds

Google, Microsoft butt heads in browser benchmark battle

Will the real real-world test please stand up?

High performance access to file storage

Google Chrome has finally met a web benchmark suite it can't master, and no one could be happier than the Internet Explorer team at Microsoft, which has used the occasion to suggest that the Chocolate Factory's browser is not really the speed demon it's cracked up to be.

Dubbed RoboHornet, the new, open source benchmark suite is different from earlier ones in that it doesn't focus on timing arbitrary JavaScript algorithms. Instead, it collects code snippets found in popular web frameworks and applications – including jQuery, YUI, Google Apps, and others – in an attempt to demonstrate real-world performance problems.

The result is a formidable benchmark suite that takes minutes to complete on typical hardware – assuming the browser can make it through the full list of tests at all.

"It's a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points," says Google Chrome product manager Alex Komoroske, who launched the project.

That a Google staffer came up with the idea for RoboHornet should surprise no one, since the ad-slinging giant has been responsible for several earlier browser benchmarks. What is surprising, on the other hand, is that Chrome doesn't always earn the highest score, depending on which you run it on.

In tests conducted by Tom's Hardware, Internet Explorer 10 scored as much as 22 per cent higher than the most recent stable version of Chrome when running on Windows 8, and Safari bested Chrome by a similar margin on OS X.

Those findings did not go unnoticed by Microsoft, but despite the strong showing by its latest browser, Redmond was not impressed – not with IE's benchmark scores and not with RoboHornet itself.

"While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn't all that representative of the performance users might encounter on real-world sites," wrote Microsoft's Roger Capriotti in a blog post on Tuesday.

To prove it, Microsoft engineers rolled their own version of RoboHornet that they're calling RoboHornet Pro, which Capriotti claims does a better job of modeling what goes on inside the browser when a real-world web app is running.

Redmond's version of the benchmark runs a Matrix-like animation in the foreground while its tests are running in the background. Although Microsoft's truncated test suite takes a lot less time to complete, the animation stutters to a halt on Chrome after a few seconds, while IE10 keeps both the animation and the benchmarks running in tandem.

According to Capriotti, that's because RoboHornet is merely the latest in a long line of browser "micro-benchmarks", which really only demonstrate isolated aspects of browser performance under lab-like conditions.

"While we will look to see how we stack up on other browser vendors' tests, our focus will remain on real-world site performance," he wrote.

Neither Komoroske or any of the other RoboHornet code stewards has commented directly on RoboHornet Pro or Microsoft's findings, but Komoroske at least acknowledges that there is more work to be done.

"RoboHornet is still in a very early alpha state, but that's where you come in," Komoroske writes. "It's up to you and to propose and vote for performance issues you care about, helping shape the future of the benchmark and effectively defining the areas that browser vendors will invest in making run faster!"

Developers who want to get involved can participate via the project's GitHub page. ®

High performance access to file storage

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Oh no, Joe: WinPhone users already griping over 8.1 mega-update
Hang on. Which bit of Developer Preview don't you understand?
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
IRS boss on XP migration: 'Classic fix the airplane while you're flying it attempt'
Plus: Condoleezza Rice at Dropbox 'maybe she can find ... weapons of mass destruction'
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
New Facebook phone app allows you to stalk your mates
Nearby Friends feature goes live in a few weeks
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.