Feeds

Fujitsu fires up first petaflopper PrimeHPC FX10

A chip off the new K

Internet Security Threat Report 2014

IT conglomerate Fujitsu has fired up the first installation of its PrimeHPC FX10 massively parallel Sparc-based supercomputer, a machine called Oakleaf-FX that weighs in at 1.13 petaflops of peak raw performance.

The University of Tokyo's Super Computing Division/Information Technology Center is the home of the Oakleaf-FX super, which has a list price of around $32m. Oakleaf-FX is being installed in the Second Research Complex on UT's Kashiwa Campus.

As El Reg detailed exhaustively last November in the wake of the SC11 supercomputer conference, the PrimeHPC FX10 is a follow-on machine to the K supercomputer built by Fujitsu for the Japanese government, sporting the eight-core Sparc64-VIIIfx processor running in a whopping 22,032 nodes (by which Fujitsu means CPU sockets not server blades since the blades do not implement SMP or NUMA clustering across the blade) and delivering a peak theoretical performance of 11.28 petaflops and a sustained performance on the Linpack Fortran benchmark of 10.51 petaflops.

Fujitsu Oakleaf-FX PrimeHPC super

Japan's Oakleaf-FX petaflopper at the University of Tokyo

For the PrimeHPC FX10 machines, Fujitsu has etched a new 16-core Sparc64-IXfx processor that runs at 1.85GHz and delivers 236 gigaflops of double-precision floating point oomph.

The Sparc64IXfx has two memory controllers on the die as well as an interface into the "Tofu" 6D mesh/torus interconnect that is at the heart of the K and PrimeHPC FX10 machines. That Tofu interface reaches out to its own Tofu interconnect chip (just like Cray talks to its XE interconnect through the HyperTransport ports on AMD's Opteron processors), which provides remote direct memory access (RDMA) across the processors in the cluster and which also provides PCI-Express 2.0 links for external storage and other peripherals. There are four Sparc64-IXfx processors and four Tofu interconnect chips per server blade.

The PrimeHPC FX machine is designed to scale up to 1,024 racks with 98,304 compute nodes, 6PB of main memory across those nodes, and a little more than 23 petaflops of peak number-crunching power, at a cost of $655.4m and with 22.4 megawatts of power consumed.

That's about half the power draw of the K super, and half the estimated budget of the original "Next Generation Supercomputer" project that resulted in the Fujitsu-only K super that is now at the top of the Top 500 supercomputer charts – NEC and Hitachi were supposed to help add vector math units and other stuff to the machine, but bagged on the Japanese government.

The University of Tokyo is not about to test the upper limits of the PrimeHPC FX10, but is nonetheless committing to a very powerful machine. The Oakleaf-FX system has 50 server racks with a total of 4,800 nodes with a total of 76,800 cores to deliver its 1.13 petaflops of raw performance. Those nodes have 150TB of main memory in total, and no, this is not a shared memory system. (Wouldn't that be a hell of a Crysis machine? It's a pity that the box is running a Sparc variant of Linux.)

The Sparc nodes in the Oakleaf-FX system are front-ended by 16 Primergy RX200S6 and 58 Primergy RX300S6 servers, which are used as log-in management nodes for the machine. The processing capacity in the log-in nodes alone would have qualified as HPC a decade ago.

In addition to running Fujitsu's homegrown Linux for Sparc, the company has Fortran, C, and C++ compilers, and a number of math libraries for common routines in what it calls the Technical Computing Suite.

The machine's storage, which consists of 150 Eternus DX80S2 storage arrays with RAID 5 across 1.1PB of capacity, and 80 Eternus DX410S2 arrays with RAID 6 across 2.1PB of capacity, runs Fujitsu's own homegrown Fujitsu Exabyte File System, or FEFS, a variant of the open source Lustre file system popular with the HPC crowd.

Fujitsu says that Oakleaf-FX was installed specifically because it is compatible with the K super. It will be used to run a slew of applications, including simulations for earth science, seismology, weather modeling, biology, hydrodynamics, astrophysics, materials science, energy, and solid-state physics. The machine will be a foundation of the application development efforts as Japan pursues an exascale super.

The University of Tokyo's current machines include two Hitachi parallel supers: an HA8000 and an SR16000. The HA8000 has been running since June 2008, and is an Opteron-based box rated at 140.1 teraflops. The SR16000 is none other than a rebadged Power 775 behemoth from IBM, and this particular machine is a 56-node box rated at 54.9 teraflops.

The Power 755 was supposed to be at the heart of the petaflops-busting "Blue Waters" super at the University of Illinois, but IBM pulled the plug on that deal last August. Last November Cray swooped in and landed a $188m contract with the university's National Center for Supercomputer Applications to build a 10 petaflops hybrid AMD Opetron-Nvidia GPU machine.

Back in Japan, the University of Tokyo has started an effort called the Large Scale HPC Challenge, a competition to seek proposals for how to use the machine. If you win the challenge, you get to run your apps on all of the nodes on the Oakleaf-FX for 24 hours straight. ®

Beginner's guide to SSL certificates

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
'Urika': Cray unveils new 1,500-core big data crunching monster
6TB of DRAM, 38TB of SSD flash and 120TB of disk storage
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
SDI wars: WTF is software defined infrastructure?
This time we play for ALL the marbles
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
Oracle hires former SAP exec for cloudy push
'We know Larry said cloud was gibberish, and insane, and idiotic, but...'
Symantec backs out of Backup Exec: Plans to can appliance in Jan
Will still provide support to existing customers
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.