US DoD plumps for Appro CPU-GPU hybrids

Supercomputer maker Appro International today announced that it has landed its first contract with the US Department of Defense and will build and install hybrid CPU-GPU machines that will be deployed in five supercomputer centers that have recently bought quite a lot of gear from Cray.

Technically speaking, Appro is a subcontractor to Lockheed Martin, and the supercomputer seller will build, integrate, and support the machines while Lockheed Martin will operate the iron and the applications running on the iron on behalf of the DoD. The systems were procured under the High Performance Computing Modernization Program that kicked off this year, which will see significant upgrades to HPC iron at the Army Research Laboratory at Aberdeen Providing Ground in Maryland; the Air Force Research Laboratory at Wright Patterson AFB in Ohio; the Army Engineer Research and Development Center in Vicksburg in Mississippi; the Navy Supercomputing Resource Center at Stennis Space Center in Mississippi; and the Arctic Region Supercomputing Center Alaska.

The new clusters have a total of 147.5 teraflops of processing power, as based on Appro's Tetra 1U servers, dual-socket boxes using X64 processors. They can cram up to four GPU co-processors into the same chassis as the CPUs. Appro introduced the Tetra servers back in May when Nvidia first started shipping its fanless M2050 and M2070 GPU co-processors for servers. According to John Lee, of vice president of advanced technology solutions at Appro, the server nodes are based on eight-core Opteron 6100 processors from Advanced Micro Devices. About a quarter of the nodes in the machines have 256 GB fat memory configurations, while the remaining nodes have the more normal 32 GB memory (for a lot of workloads, 2 GB of memory per core is the rule of thumb, says Lee).

About a third of the nodes have Nvidia M2050 GPU co-processors on them, and the GPUs represent about 83.4 teraflops of peak performance, or about 57 per cent of the number-crunch power in the machines. The servers in each of the five clusters have two-rail InfiniBand networks, for redundancy and extra bandwidth between the nodes, as well as redundant Gigabit Ethernet networks for applications that require this slower but more common networking scheme. Appro is using its own ACE management tools to run the clusters, which support Red Hat Enterprise Linux and possibly Windows HPC Server R2 because some of the labs are starting to ask for dual-boot options.

The five clusters have an aggregate of just over one hundred server nodes, and each DoD lab is getting a similar machine to ensure application compatibility. Lee says that Appro's portion of the Defense Department contract was worth a little over $7m, which works out to about $48 per gigaflops.

Back in February, Cray announced that it had inked $45m in deals with the DoD to put three of its Opteron-based XE6 machines, sporting their "Gemini" high-bandwidth interconnect, into the Air Force Research Laboratory, the Arctic Region Supercomputing Center, and the Army Engineer Research and Development Center. The DoD didn't specify how much capacity these three labs were getting, but at the time, El Reg estimated that the six labs mentioned above had around 1 petaflops of aggregate number-crunching power and the three Cray machines would come in at around 1 petaflops, at a cost of around $45 per gigaflops. This is very likely not an apples-to-apples comparison, since in neither DoD contract were the hardware, software, and services components of the deals spelled out in any detail. ®