Hoosiers to get the world's fastest academic super

Alley-oop for privately paid for petaflops

Indiana University's new data center for Big Red II

Indiana University is set to rise to the top of the flops among the world's academic institutions as it aims to break through the petaflops barrier with its upcoming "Big Red II" supercomputer using a Cray XK7 hardware - at least as gauged by raw computing power, and specifically those that foot their own bill for machines.

The XK7 has not been announced yet by Cray, but that is obviously the future name of the machine as you could see from the benchmark tests that Cray and Nvidia did recently to test the meddle (surely metal?) of the "Kepler2" Tesla K20 coprocessors that will be at the heart of the "Titan" super at Oak Ridge National Laboratory and the similar "Blue Waters" super at the Hoosiers' Big Ten football rival, the Fighting Illini of the University of Illinois.

Although Blue Waters is expected to weigh in at between 10 and 20 petaflops and be therefore considerably more powerful than the Big Red II machine at IU, Blue Waters is largely paid for through a grant from the National Science Foundation to the National Center for Supercomputing Applications (NCSA), the place where Marc Andreessen famously cut his teeth developing the Mosaic browser they led to riches for him.

The Big Red II machine will have over 21,000 processor cores and use the latest processors from Advanced Micro Devices as well as the latest Tesla GPU accelerators from Nvidia.

That almost certainly means the current "Interlagos" Opteron 6200s from AMD and the forthcoming K20 GPU coprocessors from Nvidia, but Cray was intentionally imprecise in its statement given the fact that the XK7 super is not yet announced, much less available.

We know it is not a "Cascade" machine going into IU as Big Red II since Cray says it will be using the current "Gemini" XE interconnect that is used in both the all-CPU Cray XE6 and the hybrid ceepie-geepie XK6 machines today.

The Hoosiers built a funky new data center for $32.5m in 2009 to house supers like Big Red II:

Indiana University's data center for Big Red II

The Hoosiers' data center for Big Red II (photo courtesy of Indiana University)

The original Big Red super was installed in 2006 by Indiana University and was built by Big Blue, as it turns out, not Cray. The machine is based on IBM's JS21 blade servers for its BladeCenter chassis, and in this case the nodes were based on dual-core PowerPC 970MP processors.

The 2,050 nodes in Big Red were linked with Myrinet 2000 cluster adapters and linked out to a cluster of pSeries p505 servers running IBM's General Parallel File System (GPFS) through Gigabit Ethernet links; this file system has 346TB of capacity.

The Big Red cluster ran SUSE Linux, the Moab workload manager from Adaptive Computing, and IBM's LoadLeveler workload scheduler, although why it needed two workload managers, we don;t know. Big Red's 4,100 cores had a theoretical peak performance of 40 teraflops and hit 28 teraflops on the Linpack Fortran benchmark test that is commonly used to reckon the horsepower of supers in a very general way.

IU also has a 16-node Xeon 7500 cluster called "Mason" and another cluster called "Quarry" that is a mix of BladeCenter HS21 blades and iDataPlex dx340 nodes that has a total of 2,960 cores and a peak performance of 26 teraflops.

While the flops and bragging rights are fun, just like college basketball (which the Hoosiers are legendary at) and college football (not so much), what has Indiana University excited about the current and future Big Red machines is the money it drives. IU says that the researchers who used the Big Red machine in the past six and a half years were able to secure more than $253m in grant money that drove research across many different domains.

"We foresee a similar positive impact being made by Big Red II," Brad Wheeler, vice president for information technology at Indiana University, said in a statement announcing the new machine.

"The key importance of these systems has been in the discoveries they have enabled, represented in thousands of published papers produced by the IU scientists who have used them. Big Red has been an extraordinary value for IU over its almost seven years of service, and the frontiers with Big Red II will be even more expansive."

Big Red will be retired in early 2013 when Big Red II comes in, and the older machine has already done two more years of duty than the typical super does. Neither Cray nor Indiana University divulged what the petaflopper will cost. ®

Sponsored: Webcast: Build the next generation of your business in the public cloud


Biting the hand that feeds IT © 1998–2020