Feeds

Cray nabs $45m nuke lab petaflops super deal

Panasas in the Cielo

7 Elements of Radically Simple OS Migration

The Los Alamos National Laboratory has inked a $45m contract with supercomputer maker Cray to supply the nuke researchers with one of the first Baker series of Opteron-based supers that will ship later this year - and to provide some company for the petaflopping RoadRunner Opteron-Cell massively parallel super, custom-built by rival IBM for Los Alamos.

The Baker machines will couple the latest Opteron processors from Advanced Micro Devices with Cray's new 3D torus interconnect, code-named Gemini. The name suggests that the kicker to the current SeaStar+ interconnect will double up the performance of the crossbar and therefore also double the number of nodes that can be clustered together - although Cray has thus far been pretty vague about the capabilities of the Gemini interconnect.

The HPC supplier has also said very little about what will make Baker machines distinct from its impending XT6 supers, which use the twelve-core Magny-Cours Opteron 6100 processors that were announced earlier this week. With twelve-core Opterons and twice the interconnect bandwidth, the Baker machines should be able to deliver around 3.5 petaflops of sustained performance, according to my back-of-envelope calculations.

The machine that is going into Los Alamos is to be nicknamed Cielo, presumably because the sky is the limit but also because the US National Nuclear Security Administration is frustrated by current performance ceilings as it manages the country's arsenal of 6,000 nuclear weapons.

The NNSA is a semi-autonomous agency within the Department of Energy, which makes and manages nukes for Uncle Sam. The major super labs in the States - Los Alamos, Sandia National Laboratories, and Lawrence Livermore National Laboratory - are all affiliated with the NNSA effort, which also has the goal of designing, completely within a supercomputer, new nuclear weapons and simulating their explosions. (You can't do this with real nukes because of the Nuclear Test Ban Treaty, of course.)

In its announcement, the NNSA says it chose the Baker boxes after a "highly competitive procurement process," which no doubt pitted IBM and Cray systems, and perhaps others, against each other. NNSA did not specify who was involved with the bidding, but did say that the Cielo super would be installed in the third quarter of 2010 with additional upgrades to the system expected in 2011. NNSA added that the Cielo massively parallel super could have more than ten times the performance of the ASCI Purple super installed at Lawrence Livermore, which is currently rated at 92.8 teraflops of peak performance. So we are talking petaflops territory for Cielo.

The Purple super is based on IBM's dual-core 1.9GHz Power5+ processors, has 12,208 of them and 48TB of main memory, and is lashed together using IBM's proprietary Federation interconnect for its AIX boxes. Lawrence Livermore has a BlueGene/P super rated at 501.4 teraflops and the original BlueGene/L super, currently rated at 596.4 teraflops. These machines have huge numbers of PowerPC cores and run Linux.

NNSA says that the total value of the Cielo contract will be under $54m, and it looks like Cray will get the lion's share of $45m, with some left over for other things such as storage systems. Panasas, an existing storage supplier to Los Alamos for the RoadRunner system, is supplying its ActiveStor scale-out network-attached storage (NAS), which uses clustered nodes with an overall multi-petabyte capacity to deliver files fast enough to keep the Cielo cores busy.

John Morrison, the lab's high performance computing division leader, said: "The file system provided by Panasas for Cielo will support users at all three National Nuclear Security Administration laboratories, including Lawrence Livermore National Laboratory, Los Alamos National Laboratory, and Sandia National Laboratories in their use of the system."

It's a good win for Panasas, keeping it in the public eye as it fights for business with Isilon, BlueArc, the HP-acquired Ibrix, and others.

The $45m Baker system win at NNSA follows another $45m deal Cray inked in late February with the US Department of Defense to put three Baker/Gemini systems into three different supers used by the Air Force, the Army, and the Arctic Region Supercomputing Center, which does work for the military. El Reg estimated back then that this $45m would have bought about 1 petaflops of number-crunching power for the US military across those three systems, and the price and expected performance of the Cielo machine bears out those estimates. ®

Best practices for enterprise data

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
VMware builds product executables on 50 Mac Minis
And goes to the Genius Bar for support
Multipath TCP speeds up the internet so much that security breaks
Black Hat research says proposed protocol will bork network probes, flummox firewalls
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Microsoft's Euro cloud darkens: US FEDS can dig into foreign servers
They're not emails, they're business records, says court
Microsoft says 'weird things' can happen during Windows Server 2003 migrations
Fix coming for bug that makes Kerberos croak when you run two domain controllers
Cisco says network virtualisation won't pay off everywhere
Another sign of strain in the Borg/VMware relationship?
prev story

Whitepapers

7 Elements of Radically Simple OS Migration
Avoid the typical headaches of OS migration during your next project by learning about 7 elements of radically simple OS migration.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
A new approach to endpoint data protection
What is the best way to ensure comprehensive visibility, management, and control of information on both company-owned and employee-owned devices?