Feeds

Fujitsu fires up first petaflopper PrimeHPC FX10

A chip off the new K

Boost IT visibility and business value

IT conglomerate Fujitsu has fired up the first installation of its PrimeHPC FX10 massively parallel Sparc-based supercomputer, a machine called Oakleaf-FX that weighs in at 1.13 petaflops of peak raw performance.

The University of Tokyo's Super Computing Division/Information Technology Center is the home of the Oakleaf-FX super, which has a list price of around $32m. Oakleaf-FX is being installed in the Second Research Complex on UT's Kashiwa Campus.

As El Reg detailed exhaustively last November in the wake of the SC11 supercomputer conference, the PrimeHPC FX10 is a follow-on machine to the K supercomputer built by Fujitsu for the Japanese government, sporting the eight-core Sparc64-VIIIfx processor running in a whopping 22,032 nodes (by which Fujitsu means CPU sockets not server blades since the blades do not implement SMP or NUMA clustering across the blade) and delivering a peak theoretical performance of 11.28 petaflops and a sustained performance on the Linpack Fortran benchmark of 10.51 petaflops.

Fujitsu Oakleaf-FX PrimeHPC super

Japan's Oakleaf-FX petaflopper at the University of Tokyo

For the PrimeHPC FX10 machines, Fujitsu has etched a new 16-core Sparc64-IXfx processor that runs at 1.85GHz and delivers 236 gigaflops of double-precision floating point oomph.

The Sparc64IXfx has two memory controllers on the die as well as an interface into the "Tofu" 6D mesh/torus interconnect that is at the heart of the K and PrimeHPC FX10 machines. That Tofu interface reaches out to its own Tofu interconnect chip (just like Cray talks to its XE interconnect through the HyperTransport ports on AMD's Opteron processors), which provides remote direct memory access (RDMA) across the processors in the cluster and which also provides PCI-Express 2.0 links for external storage and other peripherals. There are four Sparc64-IXfx processors and four Tofu interconnect chips per server blade.

The PrimeHPC FX machine is designed to scale up to 1,024 racks with 98,304 compute nodes, 6PB of main memory across those nodes, and a little more than 23 petaflops of peak number-crunching power, at a cost of $655.4m and with 22.4 megawatts of power consumed.

That's about half the power draw of the K super, and half the estimated budget of the original "Next Generation Supercomputer" project that resulted in the Fujitsu-only K super that is now at the top of the Top 500 supercomputer charts – NEC and Hitachi were supposed to help add vector math units and other stuff to the machine, but bagged on the Japanese government.

The University of Tokyo is not about to test the upper limits of the PrimeHPC FX10, but is nonetheless committing to a very powerful machine. The Oakleaf-FX system has 50 server racks with a total of 4,800 nodes with a total of 76,800 cores to deliver its 1.13 petaflops of raw performance. Those nodes have 150TB of main memory in total, and no, this is not a shared memory system. (Wouldn't that be a hell of a Crysis machine? It's a pity that the box is running a Sparc variant of Linux.)

The Sparc nodes in the Oakleaf-FX system are front-ended by 16 Primergy RX200S6 and 58 Primergy RX300S6 servers, which are used as log-in management nodes for the machine. The processing capacity in the log-in nodes alone would have qualified as HPC a decade ago.

In addition to running Fujitsu's homegrown Linux for Sparc, the company has Fortran, C, and C++ compilers, and a number of math libraries for common routines in what it calls the Technical Computing Suite.

The machine's storage, which consists of 150 Eternus DX80S2 storage arrays with RAID 5 across 1.1PB of capacity, and 80 Eternus DX410S2 arrays with RAID 6 across 2.1PB of capacity, runs Fujitsu's own homegrown Fujitsu Exabyte File System, or FEFS, a variant of the open source Lustre file system popular with the HPC crowd.

Fujitsu says that Oakleaf-FX was installed specifically because it is compatible with the K super. It will be used to run a slew of applications, including simulations for earth science, seismology, weather modeling, biology, hydrodynamics, astrophysics, materials science, energy, and solid-state physics. The machine will be a foundation of the application development efforts as Japan pursues an exascale super.

The University of Tokyo's current machines include two Hitachi parallel supers: an HA8000 and an SR16000. The HA8000 has been running since June 2008, and is an Opteron-based box rated at 140.1 teraflops. The SR16000 is none other than a rebadged Power 775 behemoth from IBM, and this particular machine is a 56-node box rated at 54.9 teraflops.

The Power 755 was supposed to be at the heart of the petaflops-busting "Blue Waters" super at the University of Illinois, but IBM pulled the plug on that deal last August. Last November Cray swooped in and landed a $188m contract with the university's National Center for Supercomputer Applications to build a 10 petaflops hybrid AMD Opetron-Nvidia GPU machine.

Back in Japan, the University of Tokyo has started an effort called the Large Scale HPC Challenge, a competition to seek proposals for how to use the machine. If you win the challenge, you get to run your apps on all of the nodes on the Oakleaf-FX for 24 hours straight. ®

Boost IT visibility and business value

More from The Register

next story
Pay to play: The hidden cost of software defined everything
Enter credit card details if you want that system you bought to actually be useful
HP busts out new ProLiant Gen9 servers
Think those are cool? Wait till you get a load of our racks
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
Community chest: Storage firms need to pay open-source debts
Samba implementation? Time to get some devs on the job
Like condoms, data now comes in big and HUGE sizes
Linux Foundation lights a fire under storage devs with new conference
Silicon Valley jolted by magnitude 6.1 quake – its biggest in 25 years
Did the earth move for you at VMworld – oh, OK. It just did. A lot
prev story

Whitepapers

Gartner critical capabilities for enterprise endpoint backup
Learn why inSync received the highest overall rating from Druva and is the top choice for the mobile workforce.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Rethinking backup and recovery in the modern data center
Combining intelligence, operational analytics, and automation to enable efficient, data-driven IT organizations using the HP ABR approach.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.