Feeds

Swiss boffins jump in Lake Lugano for Cray super

Pumping water uphill to cool a 750 teraflops Cascade

Combat fraud and increase customer satisfaction

The boffins at the Swiss National Supercomputing Centre (CSCS) are opened up a new supercomputing center in Lugano this year, and now they are getting a shiny new "Cascade" parallel supercomputer from Cray to hum away inside of it.

The Cascade machine at CSCS is not just interesting because of the "Aries" interconnect that will glue its x86 server nodes together so they can share work. It is also notable because it will be cooled by Lake Lugano, the glacial lake that sits on the border between Switzerland and Italy.

Cray has not announced the Cascade machines yet, but here's what we do know. The new Aries interconnect links nodes together through a much-improved high-radix router that speaks to server nodes through a PCI-Express bus.

Given that the initial Cascade machines are based on Xeon E5 processors from Intel, and that these processors have PCI-Express 3.0 controllers right on the chip, everyone expects that only processors that support PCI-Express 3.0 links will plug into the Cascade boxes. So that means for now, the Opteron 6200s from Advanced Micro Devices are outta luck.

The Aries interconnect and Cascade system design were funded by the US Defense Advanced Research Projects Agency, so it has dibs on the first machine manufactured by Cray, which is expected to be delivered later this year. The Cascade line will be commercialized in 2013.

It looks like CSCS is next in line after that, with Cray saying that its Cascade machine will be installed later this year. That machine does not yet have a nickname, but if history is any guide, it will have an Alps mountain as its namesake.

This Cascade box will have in excess of 750 teraflops of peak performance, according to Cray, which tells El Reg that this particular initial configuration will be entirely based on Xeon processors. With the top-bin Xeon E5 part weighing in at 172 gigaflops per socket, and a server node having two sockets, this Cascade machine should have around 2,180 nodes. 

The CSCS machine will not have either Intel Xeon Phi or Nvidia Tesla K20 coprocessors. These two coprocessors are, of course, options on the Cascade box, and it is reasonable to assume that at some point AMD Opterons and maybe even FirePro discrete graphics cards might also be options.

The CSCS supercomputer center in Lugano

The CSCS supercomputer center in Lugano

The CSCS super will also be equipped with Cray's Sonexion storage arrays, which run the Lustre cluster file system. The word on the street is that the machine will have north of 1PB of capacity.

The Swiss boffins say this is only the first step in building up their HPC compute capacity, which will drive research in materials science, climate modeling, geosciences, molecular biology, and biomedical engineering.

You can watch a time-lapsed video of the new data center being constructed here, which took from February 2010 through January 2011 to complete. The site was an old bus depot, and it is a concrete cube that has reconfigurable floors designed to be able to support supercomputers for the next 40 years.

At 45 meters below the surface of Lake Lugano, the water is only 6 degrees Celsius, and it is this natural source of coolness that the future Cascade super will be chilled with. The data center is 2.8 kilometers from the lake, and has to be pushed up an elevation of 30 meters from the lake level.

A little more than half of the water pumped uphill goes to the data center, while the remainder goes to fill a reservoir; the pumps can push 760 liters of water per second.

The pumps to move lake water to the CSCS data center in Lugano

The pumps to move lake water to the CSCS data center in Lugano

The cool lake water is brought into the data center, where it flows through heat exchangers, picking up the warmth from the running system. There are two pipes coming into the data center: one that can handle about 14 megawatts of cooling and the second adds another 7 megawatts.

When CSCS says it is trying to future proof, it means it. The water taken out of the data center is put into a cooling pond and is not put back into the lake until it cools down to at least 25 degrees Celsius.

Birds are going to love that pond; fish, not so much.

CSCS has been moving its computers into the new center since April, and a lot of that existing gear has Cray labels on it. The Swiss boffins were first in line to buy an XMT-2 massively threaded super back in February 2011 – called "Matterhorn," of course. The XMT-2 was commercialized by Cray as the Urika big data appliance this year.

The 20-rack "Monte Rosa" Cray XE6 system was first installed as an XT5 back in June 2009, but has had its performance more than doubled with an upgrade to an XE6 machine that is rated at 402 teraflops.

CSCS also has a two-rack XK6 hybrid CPU-GPU box, called "Tödi," that has 176 Opteron nodes with one Tesla X2090 GPU coprocessor on each node, and an IBM iDataPlex ceepie-geepie called "Castor" that has a modest sixteen Xeon 5600 nodes with two M2090 GPU coprocessors per node.

The Swiss boffins also have two smaller XE6 clusters and one aged Sun Fire cluster from back when Sun Microsystems was an independent company and cared about supercomputing.

Bootnote: Dominik Ulmer, general manager of CSCS, notified El Reg after this story ran that the 750 teraflopper will be named after "Piz Daint", after a mountain in the Val Müstair area close to the Swiss National Park.®

Combat fraud and increase customer satisfaction

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
BOFH: Oh DO tell us what you think. *CLICK*
$%%&amp Oh dear, we've been cut *CLICK* Well hello *CLICK* You're breaking up...
AMD's 'Seattle' 64-bit ARM server chips now sampling, set to launch in late 2014
But they won't appear in SeaMicro Fabric Compute Systems anytime soon
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.