Original URL: http://www.theregister.co.uk/2009/03/12/ie_8_speed_test/
Microsoft claims Firefox- and Chrome-whopping IE8 speeds
Lies, damned lies and questionable lab tests
Microsoft is touting its own research that claims Internet Explorer 8 - expected next week - is faster than Firefox 3.05 and Google's Chrome 1.0.
The report and video promise to be the first step in a campaign to win back users and developers who've drifted away to Firefox and Chrome. Opera, not mentioned in this report, has also claimed an uptick in its downloads off the back of publicity around IE8.
Typically such benchmarks are only of interest to others in the industry. Yet the type of sites tested and the language of the video suggest Microsoft is trying to reach out to ordinary customers.
The opening to its promotional video states, without any trace of irony, that: "Lots talk about lab testing that involves microbenchmarks that most consumers have never heard of."
Now Microsoft has joined their ranks.
As ever with such lab tests, though, things are never quite as they seem, and the results will prove extremely difficult to reproduce in the real world.
Microsoft's IE8 road test saw content from the 25 web sites it surfed cached, a fact that'll help speed loading for all browsers.
Also, add-ons were kept to a minimum: Microsoft ran some tests with no add-ins and as few as three in other cases - for Adobe Systems' Flash plus Microsoft's own Silverlight challenger and the older Windows Media Player (WMP).
Keeping IE8 that slim in the real world will be difficult, given the ecosystem of Microsoft partners and add-in providers swimming around Windows, IE and Microsoft in general.
Microsoft, meanwhile, has not indicated whether the final results combine both sets of add-in tests or whether its final reports draws on just the best stats. If Microsoft did combine results from both sets of test metrics, it's also not clear how the results in the report were calculated.
Finally, Microsoft said it measured page load times using "visual cues" or a combination of visual cues and the ability to interact with a page to determine if a page had completely loaded. Again, it's not clear where the results Microsoft has quoted combine these separate approaches or use them individually.
Microsoft said that relying on the "done" notification would have provided inconsistent results in some sites, particularly those that contain AJAX. Timing of the test started when the "Go" button was pressed, while the tests of 25 of the web's top sites were conducted in January this year. ®