Cell supers rule the Green 500 HPC rankings

Back in November, at the annual Supercomputing event that was held in Austin, Texas this time around, a bunch of supercomputing researchers released the semi-annual Top 500 rankings of the fastest supercomputers in the world. Now, another set of nerds has added power metrics to the list and done a sort on how efficiently the supers deliver floating-point performance. What is immediately clear from the Green 500 ranking is that performance and power efficiency do not bear much relation in supercomputing - at least not yet.

The Top 500 list of supercomputing sites ranks the sustained number-crunching performance of a supercomputer (regardless of architecture, or despite it, if you want to be more accurate) on a set of Fortran benchmarks called Linpack. The list is compiled by Erich Strohmaier and Horst Simon, computer scientists at Lawrence Berkeley National Laboratory, Jack Dongarra of the University of Tennessee, and Hans Meuer of the University of Manheim.

The list is useful in gauging the bleeding edge of supercomputing technology, the prevalence of different operation systems, server nodes, architectures, and interconnection schemes, among lots of other data. (You can read our coverage of the November 2008 Top 500 supers list here.)

The Green 500 resorting of the Top 500 list is put together by Wu-chun Feng and Kirk Cameron of Virginia Tech. This is the fourth Green 500 ranking, but only the second one that has been made public. (We covered the first public one a few months ago here.)

According to Wu and Cameron, about half the machines in the Top 500 provided measured electric usage in conjunction with their Linpack results, allowing for a simple calculation of megaflops per watt. The other half have been given an estimated power usage by Wu and Cameron, and then the list (available as an Excel spreadsheet here) is sorted for power efficiency instead of raw performance. Power on the list is measured in kilowatts, and power efficiency is measured in megaflops per watt. (This is not obvious in the sheet.)

The first thing that jumps out in this fourth Green 500 list is that machines have broken through the 500 megaflops per watt barrier. These breakers are all relatively small blade boxes made by IBM, based on its QS22 PowerXCell 8i (the latest iteration of its nine-core Cell PowerPC chip) using InfiniBand interconnect between the blades. The biggest box on the Top 500 list, the "Roadrunner" hybrid Opteron-Cell blade box that has broken the 1 petaflops performance barrier, is notable because it is delivering 444.9 megaflops per watt of computing efficiency; the machine consumes 2.48 megawatts of juice, however.

There is one caveat, though. A base QS22 blade with two Cell chips and 8 GB of memory costs $9,995, which is about four times the cost of an x64 blade. So you can either pay big electricity bills or higher server costs, or try to mix x64 and Cells and find yourself somewhere in the middle.

It wasn't all that long ago that IBM was a joke in supercomputing, and the best gear that the company could put into the field to compete against then-dominant Cray and its vector machines and a slew of smaller and clever super makers was a 3090 mainframe equipped with vector co-processors. That began to change in 1995 with the "Deep Blue" chess-playing box, which was commercialized two years later as the RS/6000 PowerParallel.

IBM also built clusters of its Power-based AIX boxes, using high-speed, proprietary interconnect, and started taking its wares into the government labs of the world. These days, IBM dominates the Top 500 list with several different architectures - including BlueGene massively parallel machines with hundreds of thousands of cores, hybrid Opteron-Cell machines, clustered AIX-Power boxes, and clustered x64 machines using as mix of processors from either Intel or Advanced Micro Devices.