Feeds

Germans pick Xeon over Power for three-petaflops super

Big Blue licks Bavarian HPC cream

  • alert
  • submit to reddit

SANS - Survey on application security programs

The Partnership for Advanced Computing in Europe (PRACE), which is upgrading its pan-European supercomputing facilities across a number of large HPC labs, has announced that it has commissioned IBM to build a 3-petaflops massively parallel super using the company's iDataPlex blade-rack hybrid machines and the next-generation of Intel Xeon processors.

This machine, known as SuperMUC, is to be installed at the Leibniz-Rechenzentrum (LRZ), which is home to a big Itanium-based Altix 4700 cluster made by Silicon Graphics. SuperMUC will be a companion to the 1-petaflops "Jugene" BlueGene/P massively parallel box that the Forschungszentrum Juelich (FZJ) in North Rhine-Westphalia paid IBM for back in February 2009.

The other PRACE big box is the "Curie" massively parallel Xeon 7500-based super made by Bull for the Commissariat a l'Energie Atomique (CEA) outside Paris. The Jugene machine has 294,912 of IBM's 850MHz PowerPC cores, and has a peak performance of just over 1 petaflops, with sustained performance on the Linpack Fortran benchmark test of 825.5 teraflops. The current CEA super in France, known as the Tera 100, has 138,368 of Intel's Xeon 7500 cores, yielding 1.25 petaflops of peak and 1.05 petaflops of sustained performance on the Linpack test. The Curie super at CEA is being built this year and has 360 nodes using current Xeon 7500 processors from Intel, rated at around 100 teraflops aggregate, with a 1.5 petaflops cluster based on future Xeons coming later this year.

The LRZ, which is located in Garching, Germany, did not divulge all of the feeds and speeds of the forthcoming SuperMUC machine, but did say it would be built from the iDataPlex tray servers in the System X product line, and not from standard System x rack servers or BladeCenter blade servers from IBM.

The LRZ added that SuperMUC will be up and running by the middle of 2012, would be rated at a peak 3 petaflops of number-crunching power, would have an aggregate of 320TB across its server nodes, and would be fed by 12PB of storage. The lab said further that the machine would use more than 110,000 cores to reach that 3-petaflops performance level.

In the picture below, Arndt Bode, chairman of LRZ (which is part of the Bavarian Academy of Sciences and Humanities), and Martin Jetter, chairman of IBM Germany, are holding up one of the tray servers from the future SuperMUC machine:

SuperMUC Supercomputer Server Node

You can't see a lot from this picture, of course, but it looks like it's a two-socket tray server. There's a lot of speculation that the server nodes in SuperMUC will be based on the forthcoming "Sandy Bridge" Xeon processors, which Intel is cooking up in a 32-nanometer process.

Server versions of the Sandy Bridge chips are expected in the third quarter of 2011 for EP variants for two-socket boxes with relatively high clock speeds and low numbers of memory slots, and maybe in the fourth quarter for EX variants for two-socket, four-socket, and eight-socket boxes, generally with lower clock speeds but much more main memory per socket. Intel is getting ready to launch the PC versions of the Sandy Bridge chips this week at the Consumer Electronics Show in Las Vegas.

It seems highly unlikely that SuperMUC would be based on the ten-core "Westmere-EX" chip due in the second quarter of this year, presumably to be called the Xeon 7600 and socket-compatible with the eight-core Xeon 7500 processors. If LRZ had chosen this 32nm chip, it could have its box up and running probably in the third quarter of this year, not the middle of 2012.

As El Reg explained back in June, an eight-core Sandy Bridge Xeon chip for servers running at 3GHz would be able to do eight double-precision flops per clock, or 192 gigaflops per socket. PRACE said the SuperMUC machine would have more than 14,000 processors and more than 110,000 cores, so we know they are using a Xeon chip with eight cores. At 3.3GHz, that's 211 gigaflops per socket. If you have 84 racks of iDataPlex machines, which each have 84 servers with two cores each, that's 14,112 sockets and 2.98 petaflops of peak performance. That sure fits the data pretty well.

But there is a decent chance that LRZ might skip a generation and go straight to the "Ivy Bridge" 22-nanometer shrink of the Sandy Bridge Xeon server chips. It is expected that both Sandy Bridge and Ivy Bridge will share the LGA 2011 socket, so the iDataPlex platform would be the same either way. But with a 22nm shrink to Ivy Bridge, the chips would run a lot cooler for an eight-core version compared to Sandy Bridge, and heat is a big deal for HPC centers.

If Intel can get Ivy Bridge kickers to LRZ early – much as CEA got a slew of early Xeon 7500s to build another of the PRACE machines last year with Bull – then there's an outside possibility that the SuperMUC machine could use a 3.3GHz Ivy Bridge part and get the same performance, possibly pushing down from a 130-watt Sandy Bridge part to a 95-watt or even an 80-watt Ivy Bridge part – those are my guesses for the kind of energy savings you might see in the shrink from 32nm to 22nm, and saving a load of energy consumption and heat.

What IBM is saying for sure is that SuperMUC will have water blocks on processors and main memory on the iDataPlex system boards, and a closed-loop water-cooling system that will use relatively warm water (up to 45 degrees Celsius, which is 113 degrees Fahrenheit) to keep these active components from melting. IBM installed a similar cooling system on a Cell-based baby super called Aquasar at the Eidgenössische Technische Hochschule back in July 2010, which can cool the machine with water that is as hot as 65 degrees Celsius.

The future BlueGene/Q massively parallel super, which El Reg told you about here, and the Power7 IH supercomputer node that is part of the "Blue Waters" parallel super, which we told you about here, both have direct water-cooling on the CPUs and memory. The Power7 IH node used in Blue Waters, which is being installed at the University of Illinois, also has water blocks on the switch/hub networking components in the node.

The SuperMUC's 10PB disk array will run IBM's Global Parallel File System.

The German federal government and the state of Bavaria are kicking in €83m (£71.6m, $110.9m) to build SuperMUC at LRZ; that figure includes the cost of the hardware and software as well as operational costs, including the electricity bill, for five years. The Federal and state governments are also ponying up €50m (£43.1m, $66.8m) to build out the data centers at the LRZ facility in Garching to house the machinery.

The SuperMUC machine will be a serious upgrade for LRZ, and you have to believe that SGI was fighting hard to have the German HPC center upgrade its 62.3-teraflops Altix 4700, called HLRB-II, to a petaflops-class "UltraViolet" Altix UV super using Intel's Xeon 7500s and SGI's NUMAlink 5 interconnect. But for whatever reason, LRZ decided on the iDataPlex alternative using Xeon chips and a future InfiniBand switching technology called FDR, short for Fourteen Data Rate and offering 56GB/sec of bandwidth compared to the 40GB/sec Quad Data Rate (QDR) speed available today. ®

3 Big data security analytics techniques

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
IBM rides nightmarish hardware landscape on OpenPOWER Consortium raft
Google mulls 'third-generation of warehouse-scale computing' on Big Blue's open chips
It's GOOD to get RAIN on your upgrade parade: Crucial M550 1TB SSD
Performance tweaks and power savings – what's not to like?
AMD's 'Seattle' 64-bit ARM server chips now sampling, set to launch in late 2014
But they won't appear in SeaMicro Fabric Compute Systems anytime soon
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.