'Real' JavaScript benchmark topped by...Microsoft

Google Chrome dead last

Douglas Crockford - the man who "discovered" JSON and a senior JavaScript architect at Yahoo! – has released a new benchmark designed to test the "actual" performance of the major web browsers on "real" JavaScript applications. And according to the test, the browser with the speediest JavaScript engine...is not Google Chrome.

In fact, when running the benchmark, Chrome and its new Crankshaft engine are slower than all the other major browsers. And the fastest browser, in the estimation of Crockford, isn't Firefox or Opera.

It's Internet Explorer 10, which is currently available as a preview.

A recent paper coauthored by Microsoft researchers, Crockford says, underscores the fact that existing JavaScript benchmarks fail to provide an accurate measure of performance. "JSMeter: Measuring JavaScript Behavior in the Wild (PDF) showed that benchmarks are not representative of the behavior of real web applications," he writes. "But lacking credible benchmarks, engine developers are tuning to what they have. The danger is that the performance of the engines will be tuned to non-representative benchmarks, and then programming styles will be skewed to get the best performance from the mistuned engines."

His benchmark is based on JSLint, a JavaScript code quality tool developed by Crockford himself. It's designed to look for problems in JavaScript programs, and it too is a JavaScript program. Unlike existing JavaScript benchmarks, Crockford says, his is more representative of large, well-written JavaScript programs – because JSLint is a large well-written JavaScript program.

"Size may matter when considering optimization strategies," Crockford tells The Register. "JSLint does a lot of what JavaScript applications do, including regular expressions, string building, and use of prototypal and functional patterns.

"JSLint is a code quality tool, and is itself the product of a code quality tool."

The benchmark runs JSLint on its main source file, jslint.js, using the "Good Parts" options. And Crockford believes the results are "more indicative of actual JavaScript engine performance" than those you get from existing benchmarks.

Asked if benchmarks should measure crap JavaScript code as well as well-written applications, Crockford says no. "There is indeed a lot of crap JavaScript out there, and most of it does not benefit from the faster JavaScript engines because performance of those applications is limited by the DOM (Document Object Model, the browser's crap API)," he tells us.

"The faster JavaScript engines are interesting because they will enable new kinds of applications to be written in JavaScript."

His results puts Internet Explorer 10 at the top of the list and Chrome at the very bottom, below Internet Explorer 9. Firefox is the second fastest, just ahead of Safari:

Douglas Crockford JavaScript benchmark

Crockford admits that he expected Chrome to top the list. "My guess is that they overspecialized for specific styles of programming, and that Chrome was tripped up by a real program. There are some very smart people at Google, and I would expect them to rectify this."

Chrome 10, the Google browser tested by Crockford, marked the debut of Crankshaft, a new JavaScript engine billed as providing a 66 per cent speed improvement. Crankshaft was inspired by Sun's Java HotSpot performance engine, according to an exchange between The Register and Google engineer Eric Kay in December.

Crankshaft uses "adaptive compilation", identifying important or "hot" code and working to optimize that code. In addition to a base compiler, it includes a runtime profile that identifies hot code, and an optimizing compiler that recompiles the hot code to offer such optimizations as loop-invariant code motion, linear-scan register allocation, and inlining. Google also provides "deoptimization support", identifying cases where the optimizing compiler has promised too much optimization. In this case, the engine falls back to the base compiler.

The approach is similar to what Firefox has done with the TraceMonkey extension to its JavaScript engine. TraceMonkey does not use adaptive compilation, but it works to detect code loops and covert them to assembly code.

But neither can match IE10. At least in the estimation of Douglas Crockford, a third-party with an appropriately impressive CV. ®

Sponsored: 10 ways wire data helps conquer IT complexity