World's 'most green' supercomputer in red-hot battle between Intel, Nvidia
Uni boffins demand more bang for their watt
Analysis Non-profit consortium CINECA has deployed what may be the greenest supercomputer in the world at its Bologna centre in Italy. Called Eurora, the new machine claims it can perform 3,150 megaflops per watt, compared to the 2,499.44 achieved by Green-500 king the Beacon supercomputer at the National Institute for Computational Sciences and University of Tennessee.
CINECA is owned by 57 Italian universities and other institutions, and has the largest computing centre in Italy.
Eurora was built by Eurotech and is a prototype of its Aurora Tigon product range. The supercomputer consists of 64 nodes, each of which has two Intel Xeon E5-2687W CPUs, two Nvidia Tesla K20 GPU accelerator cards, an Altera Stratix V FPGA, 16GB DDR3 RAM and 160GB in solid-state storage.
According to Eurotech, each node, similar in size to a laptop, is 30 times more powerful and 15 times more energy-efficient than a desktop PC. Each node is capable of 1,700 gigaflops per second.
High energy-efficiency is achieved in various ways. The most obvious is water cooling. Each node is built like a sandwich: the main board is underneath, a water-cooled heat sink sits in the middle, and the accelerator boards are on top. RAM is soldered to the board rather than socketed. The power supply is 48 volts DC, minimising power conversion steps. Water heated by the system can be converted to energy for other purposes.
The water-cooling system
Underneath Eurora is a lot of pipework, leading to a pump and heat extraction unit. When you plug a node into the rack, it makes several fluid connections for cooling along with the usual electric connections, though Eurotech assured us that failsafe devices prevent the possibility of a spill. The metal of the heat sink is also effective in absorbing heat, allowing time for clean shutdown in the event of a pump failure.
Behind all the green talk at the Bologna press event is another battle, though. Nvidia is fighting Intel for share of the hybrid computer market. A hybrid computer is one that combines traditional CPUs with accelerator boards packed with many simpler processing cores, all of which achieves more power-efficient parallel processing. This was a market Nvidia had almost to itself with its Tesla GPU boards, but Intel now offers its own Many Integrated Core (MIC) product, called Xeon Phi.
Each Xeon Phi core is x86 compatible, which makes it more familiar to programmers - although Nvidia says that its Cuda C programming language for its own chips is now popular in its own right. After a long preview period, Xeon Phi is now generally available.
“We’re very competitive in performance,” Intel multi-core evangelist James Reinders told us. “We have higher performance bandwidth. We have the best power efficiency. But programmability is the key. Because it is x86-based and we boot Linux on the cards, you can run whatever you want there.”
Nvidia says its GPUs are more power-efficient, but given that Beacon topped the Green 500 with a Xeon Phi machine, the advantage cannot be overwhelming.
It turns out that Eurora will be a key test for Nvidia’s solution versus that from Intel. Eurora is called a prototype because it will be used for a future tender, planned to begin in 2014, for a new CINECA supercomputer.
Eurora itself will use Xeon Phi coprocessors as well as the Nvidia K20 GPU accelerators, this being a feature of the Tigon nodes. Each node supports either chip family, and one can be replaced with another. Currently Eurora has 128 K20 boards, since the Xeon Phi is not yet ready in the right small-form factor, but in a few months half those K20s may be replaced with Xeon Phi, or alternatively Xeon Phi nodes may be added, depending on what budget is available to CINECA.
“We have to go through a public tender. We cannot prefer a particular technology, we have to be open, because we are investing public money,” says CINECA’s Carlo Cavazzoni. “We need to test the technology in order to collect our own data, based on our own workload. From what we have seen up to now, the Kepler card has advantages in terms of pure performance, and Intel MIC on programmability. It is by far more easy to port an application for Xeon Phi. But this is a long story in HPC [High Performance Computing].”
Might CINECA’s eventual tender include both Nvidia and Intel accelerator boards, like Eurora? “It is easier to manage a solution based on a single technology,” says Cavazzoni. “Probably at the end we have to choose.” ®