'Real' JavaScript benchmark topped by...Microsoft
Google Chrome dead last
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
Douglas Crockford - the man who "discovered" JSON and a senior JavaScript architect at Yahoo! – has released a new benchmark designed to test the "actual" performance of the major web browsers on "real" JavaScript applications. And according to the test, the browser with the speediest JavaScript engine...is not Google Chrome.
In fact, when running the benchmark, Chrome and its new Crankshaft engine are slower than all the other major browsers. And the fastest browser, in the estimation of Crockford, isn't Firefox or Opera.
It's Internet Explorer 10, which is currently available as a preview.
A recent paper coauthored by Microsoft researchers, Crockford says, underscores the fact that existing JavaScript benchmarks fail to provide an accurate measure of performance. "JSMeter: Measuring JavaScript Behavior in the Wild (PDF) showed that benchmarks are not representative of the behavior of real web applications," he writes. "But lacking credible benchmarks, engine developers are tuning to what they have. The danger is that the performance of the engines will be tuned to non-representative benchmarks, and then programming styles will be skewed to get the best performance from the mistuned engines."
His benchmark is based on JSLint, a JavaScript code quality tool developed by Crockford himself. It's designed to look for problems in JavaScript programs, and it too is a JavaScript program. Unlike existing JavaScript benchmarks, Crockford says, his is more representative of large, well-written JavaScript programs – because JSLint is a large well-written JavaScript program.
"Size may matter when considering optimization strategies," Crockford tells The Register. "JSLint does a lot of what JavaScript applications do, including regular expressions, string building, and use of prototypal and functional patterns.
"JSLint is a code quality tool, and is itself the product of a code quality tool."
The benchmark runs JSLint on its main source file, jslint.js, using the "Good Parts" options. And Crockford believes the results are "more indicative of actual JavaScript engine performance" than those you get from existing benchmarks.
Asked if benchmarks should measure crap JavaScript code as well as well-written applications, Crockford says no. "There is indeed a lot of crap JavaScript out there, and most of it does not benefit from the faster JavaScript engines because performance of those applications is limited by the DOM (Document Object Model, the browser's crap API)," he tells us.
"The faster JavaScript engines are interesting because they will enable new kinds of applications to be written in JavaScript."
His results puts Internet Explorer 10 at the top of the list and Chrome at the very bottom, below Internet Explorer 9. Firefox is the second fastest, just ahead of Safari:
Crockford admits that he expected Chrome to top the list. "My guess is that they overspecialized for specific styles of programming, and that Chrome was tripped up by a real program. There are some very smart people at Google, and I would expect them to rectify this."
Chrome 10, the Google browser tested by Crockford, marked the debut of Crankshaft, a new JavaScript engine billed as providing a 66 per cent speed improvement. Crankshaft was inspired by Sun's Java HotSpot performance engine, according to an exchange between The Register and Google engineer Eric Kay in December.
Crankshaft uses "adaptive compilation", identifying important or "hot" code and working to optimize that code. In addition to a base compiler, it includes a runtime profile that identifies hot code, and an optimizing compiler that recompiles the hot code to offer such optimizations as loop-invariant code motion, linear-scan register allocation, and inlining. Google also provides "deoptimization support", identifying cases where the optimizing compiler has promised too much optimization. In this case, the engine falls back to the base compiler.
The approach is similar to what Firefox has done with the TraceMonkey extension to its JavaScript engine. TraceMonkey does not use adaptive compilation, but it works to detect code loops and covert them to assembly code.
But neither can match IE10. At least in the estimation of Douglas Crockford, a third-party with an appropriately impressive CV. ®
COMMENTS
consider the article?
Crockford wasn't involved in the original MS co-authored paper, he merely took its findings regarding benchmarking and used his own js application as a benchmark. I'm involved in a couple of projects which are pretty js heavy and like you generally find that Chrome is markedly faster, closely followed by FF. Crockford's results are interesting with regard to IE10 because it has only just been released as a platform preview. It will be interesting to see how it shapes up at full release - all good from my point of view, healthy competition driving the technology forward.
Crockford's real-world vs. the real-real-world
Let me cut out Mr. Crockford's bias against DOM from this quote, and get it to the part that matters:
"...most of it does not benefit from the faster JavaScript engines because performance of those applications is limited by the DOM..."
This little gem is spot on, and exactly why his test is nothing like a real-world javascript program. The reality is, from the perspective of working with (X)HTML or XML (what the majority of javascript does) the DOM is how much of it is done. Because of that, a truly realistic representation of JS performance also should test out the specific implementation of the DOM.
He goes on to say:
"My guess is that they overspecialized for specific styles of programming, and that Chrome was tripped up by a real program. There are some very smart people at Google, and I would expect them to rectify this."
Yes, probably "overspecialized" for more DOM intensive styles. You know, like most of the JS that will be encountered in the real-real-world (as opposed to the "real-world" where code analyzers are the majority of the code we run *boggle*).
As impressive as Mr. Crockford's CV may be, he has still missed the forest for the trees. He has created (and I propose we call this) "Yet Another Worthless Browser Benchmark" because of his particular gripes with DOM.
Well there's your problem!
"His benchmark is based on JSLint, a JavaScript code quality tool developed by Crockford himself. It's designed to look for problems in JavaScript programs, and it too is a JavaScript program. Unlike existing JavaScript benchmarks, Crockford says, his is more representative of large, well-written JavaScript programs – because JSLint is a large well-written JavaScript program."
I don't buy it.
He says that his benchmark is more "real" because it represents large, well-written JavaScript programs because it's based on his code. I'll let the second assumption (i.e, that his code is well-written) stand because I don't intend to refute it. Instead I intend to use it against him.
Instead, I'll focus on the much larger initial assumption: that "real" JavaScript is large, well-written code. The "large" part may well be true, but if you look at JavaScript in the wild, "well-written" is very debatable.
If he wanted to put forth his benchmark as one valid for testing "proper" JavaScript performance, then we could just look at the second assumption for validity. But the first assumption is so far at odds with the prevailing evidence that I believe that it is Mr. Crockford's responsibility to prove his point.
Show us, Mr. Crockford, with a valid study proving that the majority of JavaScript out there being consumed on the web today is well written. Please. I would love to believe that to be true.
IT infrastructure monitoring strategies
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
Data control in the cloud
Cloud based data management
Enabling efficient data center monitoring
