Google Go strikes back with C++ bake-off
'Benchmarks only as good as the programs they measure'
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
In early June, Googler Robert Hundt published a paper comparing the performance of four programming languages: C++, Java, Scala, and a rather new addition to the world of systems programming, Google's own Go. Go is designed to provide the performance of a compiled language like C++ and the "feel" of a dynamic language like Python, but under Hundt's tests, its performance lagged well behind that of Java and Scala as well as C++.
At the time, one Google Go team member indicated the language didn't exactly receive a fair shake. And now, the Go braintrust has taken another a crack, using Go's profiling tools to better optimize Hundt's test program. The results are quite different. If nothing else, they show that comparing the performance of various programming languages is not an exact science – and that there's always room for debate.
Identifying and correcting bottlenecks in Hundt's Go program, Goolger Russ Cox and team improved its performance by an order of magnitude while using less than one-sixth of the memory. Then, they made the same changes to Hundt's C++ program, and in the end, the two programs ran at similar speeds.
"This shows that, despite its youth, Go is competitive with the other languages presented in the paper," Google Go man Andrew Gerrand tells The Register." And we have barely started optimizing our compiler..."
For their tests, the team used a snapshot of the 6g Go compiler and the GNU C++ compiler that ships with Ubuntu Natty Narwal. It's unclear which versions Hundt used. The Go team did not test Scala or Java, Cox says, because "we are not skilled at writing efficient programs in either of those languages, so the comparison would be unfair".
Hundt's tests allowed for optimization, but after his paper was published, Go team member Ian Lance Taylor said that very little work went into the Go optimization. "Despite the name, the [ostensibly optimized version of the Go] code was never intended to be an example of idiomatic or efficient Go. Robert [Hundt] asked me to take a look at his code and I hacked on it for an hour to make a little bit nicer. If I had realized that he was going to publish it externally, I would have put a lot more time into making it nicer," Taylor said on the Go mailing list.
Hundt's original (unoptimized) C++ and Go benchmark programs run a particular loop-finding algorithm. In revisiting the tests, Cox and his team ran these programs on a 2.13GHz Core i7 machine with 4GB of RAM running Ubuntu. The C++ code ran in 27.47 seconds and used 700MB of memory, while the Go program ran in 56.92 seconds and used 1604MB of memory. But then they optimized each piece of code.
After fine-tuning Hundt's Go program using the language's profiling tools – gopprof – they dropped its runtime to 3.84 seconds, and the program used only 257MB of memory. Then they translated the optimized code into C++ code, and according to Cox, the Go program ran slightly faster – though the C++ program was slightly shorter and easier to write because the C++ code uses automatic deletes and allocation instead of a cache.
"Benchmarks are only as good as the programs they measure," Cox said. "We used gopprof to study an inefficient Go program and then to improve its performance by an order of magnitude and to reduce its memory usage by a factor of six. A subsequent comparison with an equivalently optimized C++ program shows that Go can be competitive with C++ when programmers are careful about how much garbage is generated by inner loops." ®
COMMENTS
Of course you can!
Otherwise how would anyone make money from this computing business?
I mean Windows and Office have approximately bugger all new features compared to 10 years ago for a good 80% of their users. I fell back on an old 3.11 / Office 6 laptop a year or two ago to get a report written and- aside from the crappy black-and-white DSTN screen- I didn't notice a huge difference. It still does styles, it still has the same fonts, it still has the same font/pragraph dialogues and it still does word count. What else do most people need?
I'm really not sure what makes up all the bloat in modern software.
56.92s to 3.84sec? 1604Mb to 275Mb?
Reducing the Go code from 56.92sec to 3.84? That's a 15x improvement! And 1604Mb to 275 = 6x better. Unless the original Go code was rather inefficient, this almost sounds more like they changed the fundamental algorithms used rather than merely tweaking the code. And as Paul Shirley said above, porting this back to C++ is not a useful comparison since specific optimisations chosen for the Go version may well do nothing in C++.
But in any event, these new benchmarks only tell us how fast this particular algorithm can be made to run if you are intimately familiar with Go. I'd rather know how it is to write "good-yet-fast" code. Personally, I find C++ fairly easy to write decent and fast code but hard / nasty to write very fast code. C tends to be speedier but a bit klunkier, Java a bit easier than either but a little slower for some things and a lot slower for others, and C# better than Java on both counts but not as fast as C/C++. I'm inclined to believe the original Go results that suggest Go is not there yet with efficiency of compiler-outputted code (Java got a lot better on this score over the years), so 'normal'/easy code is slower. You can of course tailor your code to do the compiler tricks yourself, in which case it will run fast as well, but who wants to do that every time? Especially when you'll have to maintain the mess later!
This is getting tiresome.
Those of you who have been following along with us commentards for a while will know I prefer K&R C with inline assembler for serious coding. But I'm a hardware guy.
I also do COBOL, FORTRAN, Forth, Smalltalk, perl, (ba)sh, yadda, yadda ...
But the basic bottom line is that bricks are bricks. All are used to build structures. Some are small and red, and used to build small red structures (your house, perhaps). Some are slightly larger and grey, and used to build largish grey structures (your dorm, perhaps). Some are longer, and made of iron, and used to build skyscrapers. Some are irregular, and used to build massive structures like Machu Pikchu. Some are more massive, and used to build even more massive structures like Egypt's pyramids ... And some are even more massive, like Ada ;-)
ALL of them have their place. Playing one off the other is a fool's errand ...
IT infrastructure monitoring strategies
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
Data control in the cloud
Cloud based data management
Enabling efficient data center monitoring
