Original URL: http://www.theregister.co.uk/2011/11/14/bsc_arm_gpu_hybrid_supercomputer/

Spaniards prototype ARM-GPU hybrid supercomputer

Ubuntu Linux spans Tegra3-Fermi mashup

By Timothy Prickett Morgan

Posted in HPC, 14th November 2011 21:22 GMT

SC11 The Barcelona Supercomputing Center in Spain has been an enthusiastic supporter of using IBM's Power970MP blade servers as a foundation for its "MareNostrum" massively parallel Linux cluster, which fills 44 server racks and delivers 94.1 teraflops of number-crunching punch.

A few years back, this was a pretty big box, but these days, when HPC vendors are building 10 petafloppers, MareNostrum is looking a little long in the tooth. But the Spanish boffins are looking even higher to exascale computing, and are prototyping a new system called Mont Blanc that will potentially offer less expensive computing than is possible with x86 or Power processors by combining Nvidia's Tegra3 ARM processors and mobile GeForce GPU coprocessors.

The Mont Blanc project was funded in October 2011 with €14.5m and will run until 2014, experimenting with various low-power processor and GPU combinations to try to do more efficient supercomputing as a foundation for what the European Union hopes will be some clever exascale designs – perhaps even using technology developed in Europe.

The objective of the project, which you can read about here, is to build a prototype that could be scaled up to 50 petaflops within a 7 megawatt power envelope, and also the software stack that uses this system effectively.

Mont Blanc also has a second objective of designing a larger 200 petaflops system that can run on 10 megawatts and porting software to run upon it. The first large-scale machine at 50 petaflops would have to be 3.5 times more efficient that IBM's forthcoming "Sequoia" BlueGene/Q super, and anywhere from five to 10 times more efficient than other CPU or hybrid CPU-GPU machines out there today. It is a tall order.

The first prototype coming out of the BSC Mont Blanc project is built from embedded system boards based on Nvidia's Tegra2 system-on-chip (SoC) processors, which were announced last year for smartphones and tablets. Here's what it looks like:

BSC Mont Blanc prototype 1

BSC's all-ARM parallel super prototype (click to enlarge)

These Tegra2 chips are based on ARM Holding's 32-bit Cortex-A9 processors and the system board that uses them is called the Q7 embedded system. It has 1GB of DDR2 memory, a single Gigabit Ethernet port, and delivers around 2 gigaflops of number-crunching power for about 4 watts of heat. A server rack that is 1U high can hold eight of these boards, and delivers 16 gigaflops of oomph for about 35 watts of juice. A rack has ten 48-port Gigabit Ethernet switches and 256 of the Tegra2 chips, delivering 512 gigaflops of grunt for about 1.7 kilowatts or about 300 megaflops per watt.

This ain't so great in terms of power efficiency, but it is still necessary to build such a machine to get the software stack in order.

The newest prototype cooked up by BSC is much more promising and shows that ceepie-geepies are the wave of the future. This on is based on a quad-core Tegra3 SoC, which was just announced last week and which has four Cortex-A9 cores plus a fifth management core. Here's what the second BSC prototype looks like:

BSC Mont Blanc prototype 2

BSC's Tegra3-Fermi hybrid parallel super prototype (click to enlarge)

The second prototype puts the Tegra3 into the Q7 embedded system module, with the processors running at 1.5GHz and delivering around 6 gigaflops of math; it has 4GB of main memory on the board and a Gigabit Ethernet port and burns about 4 watts. Now, slide in a GeForce GT520MX mobile GPU coprocessor, which has 48 cores (9 percent of the total cores) spinning at 900MHz (instead of 1.3GHz) but which delivers 142 gigaflops of floating point math. Put eight of those in that 1U racker, and slide it into the same rack with the ten Gigabit Ethernet switches. Now you have a rack with 1,024 ARM cores and 256 GPUs with a total of 12,288 cores. With 1TB of total memory, this rack of ARM-Fermi cores has 38 teraflops of performance at around 5 kilowatts of power drawn, or 7.5 gigaflops per watt. IBM's BlueGene/Q prototype is delivering 2.1 gigaflops per watt.

I think we have a winner.

Now here's the fun bit. You don't have to be BSC to play. The system board that BSC is using is now available as a development system from Italian embedded system board maker Seco. Here's what it looks like:

Nvidia Seco ARM GPU board

The Seco ARM-GPU board, based on Nvidia chips (click to enlarge)

There are no list prices for this baby, but the word on the street at SC11 is that you can get one for around $500. ®