Feeds

Swiss boffins jump in Lake Lugano for Cray super

Pumping water uphill to cool a 750 teraflops Cascade

HP ProLiant Gen8: Integrated lifecycle automation

The boffins at the Swiss National Supercomputing Centre (CSCS) are opened up a new supercomputing center in Lugano this year, and now they are getting a shiny new "Cascade" parallel supercomputer from Cray to hum away inside of it.

The Cascade machine at CSCS is not just interesting because of the "Aries" interconnect that will glue its x86 server nodes together so they can share work. It is also notable because it will be cooled by Lake Lugano, the glacial lake that sits on the border between Switzerland and Italy.

Cray has not announced the Cascade machines yet, but here's what we do know. The new Aries interconnect links nodes together through a much-improved high-radix router that speaks to server nodes through a PCI-Express bus.

Given that the initial Cascade machines are based on Xeon E5 processors from Intel, and that these processors have PCI-Express 3.0 controllers right on the chip, everyone expects that only processors that support PCI-Express 3.0 links will plug into the Cascade boxes. So that means for now, the Opteron 6200s from Advanced Micro Devices are outta luck.

The Aries interconnect and Cascade system design were funded by the US Defense Advanced Research Projects Agency, so it has dibs on the first machine manufactured by Cray, which is expected to be delivered later this year. The Cascade line will be commercialized in 2013.

It looks like CSCS is next in line after that, with Cray saying that its Cascade machine will be installed later this year. That machine does not yet have a nickname, but if history is any guide, it will have an Alps mountain as its namesake.

This Cascade box will have in excess of 750 teraflops of peak performance, according to Cray, which tells El Reg that this particular initial configuration will be entirely based on Xeon processors. With the top-bin Xeon E5 part weighing in at 172 gigaflops per socket, and a server node having two sockets, this Cascade machine should have around 2,180 nodes. 

The CSCS machine will not have either Intel Xeon Phi or Nvidia Tesla K20 coprocessors. These two coprocessors are, of course, options on the Cascade box, and it is reasonable to assume that at some point AMD Opterons and maybe even FirePro discrete graphics cards might also be options.

The CSCS supercomputer center in Lugano

The CSCS supercomputer center in Lugano

The CSCS super will also be equipped with Cray's Sonexion storage arrays, which run the Lustre cluster file system. The word on the street is that the machine will have north of 1PB of capacity.

The Swiss boffins say this is only the first step in building up their HPC compute capacity, which will drive research in materials science, climate modeling, geosciences, molecular biology, and biomedical engineering.

You can watch a time-lapsed video of the new data center being constructed here, which took from February 2010 through January 2011 to complete. The site was an old bus depot, and it is a concrete cube that has reconfigurable floors designed to be able to support supercomputers for the next 40 years.

At 45 meters below the surface of Lake Lugano, the water is only 6 degrees Celsius, and it is this natural source of coolness that the future Cascade super will be chilled with. The data center is 2.8 kilometers from the lake, and has to be pushed up an elevation of 30 meters from the lake level.

A little more than half of the water pumped uphill goes to the data center, while the remainder goes to fill a reservoir; the pumps can push 760 liters of water per second.

The pumps to move lake water to the CSCS data center in Lugano

The pumps to move lake water to the CSCS data center in Lugano

The cool lake water is brought into the data center, where it flows through heat exchangers, picking up the warmth from the running system. There are two pipes coming into the data center: one that can handle about 14 megawatts of cooling and the second adds another 7 megawatts.

When CSCS says it is trying to future proof, it means it. The water taken out of the data center is put into a cooling pond and is not put back into the lake until it cools down to at least 25 degrees Celsius.

Birds are going to love that pond; fish, not so much.

CSCS has been moving its computers into the new center since April, and a lot of that existing gear has Cray labels on it. The Swiss boffins were first in line to buy an XMT-2 massively threaded super back in February 2011 – called "Matterhorn," of course. The XMT-2 was commercialized by Cray as the Urika big data appliance this year.

The 20-rack "Monte Rosa" Cray XE6 system was first installed as an XT5 back in June 2009, but has had its performance more than doubled with an upgrade to an XE6 machine that is rated at 402 teraflops.

CSCS also has a two-rack XK6 hybrid CPU-GPU box, called "Tödi," that has 176 Opteron nodes with one Tesla X2090 GPU coprocessor on each node, and an IBM iDataPlex ceepie-geepie called "Castor" that has a modest sixteen Xeon 5600 nodes with two M2090 GPU coprocessors per node.

The Swiss boffins also have two smaller XE6 clusters and one aged Sun Fire cluster from back when Sun Microsystems was an independent company and cared about supercomputing.

Bootnote: Dominik Ulmer, general manager of CSCS, notified El Reg after this story ran that the 750 teraflopper will be named after "Piz Daint", after a mountain in the Val Müstair area close to the Swiss National Park.®

Reducing security risks from open source software

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.