Supercomputing past masters resurface with coder-friendly cluster

Memory architecture secret sauce

Another secret sauce in the HC-1 is the memory architecture. The FPGA and its personality are plunked on a chip that has 16 memory channels reaching out to the system, providing 80 GB/sec of bandwidth into the FPGA. The x64 processor and the FPGA are linked together with a cache-coherent shared virtual memory space, and applications see the x64 instruction set and a set of co-processor instructions implemented in the FPGA's personality.

Programmers using standard C, C++, and Fortran compilers will be able to see these extra instructions implemented in the FPGA and can make use of them in their code. Convey has spent a lot of time making the debugging for the x64 and FPGA co-processor environment seamless, says Toal, and the resulting applications created using the HC-1 applications can run on regular x64 servers or on the HC-1. Applications do require a Convey-enhanced (and Linux Standard Base-compliant) Linux kernel to make use of the FPGA co-processors and their personalities.

The HC-1 system boards have four DIMM channels for the single x64 processor and 16 DIMM channels for the FPGA, which are linked to each other through the front side bus architecture (as x64 processors are in two-socket machines) and, in the future, through the QPI point-to-point interconnect. The Convey machine uses standard DIMMs that have been optimized for cache line transfers (sequential access) and also has a special set of scatter-gather DIMMs that have been optimize for 8-byte transfers (random access). The Xeon side of the system can support up to 32 GB of main memory for applications using 8 GB DIMMs, while the co-processor side of the system board can support up to 128 GB of memory.

The development tools created by Convey for the HC-1 are derived from the Open64 compiler set that is available for Itanium processors, which are themselves open source versions of the tools created by Silicon Graphics for its MIPS supercomputers. The Open64 tools have been ported to the x64 architecture and extended with lots of goodies by contributors in industry and academia. The HC-1 development tools also include features so customers can create their own custom personalities for the FPGAs, tuned specifically to accelerate calculations in their own applications.

With supercomputers, people want to know about performance. And here's the real reason why Wallach, Toal, Convey's investors, and the company's 33 employees are all excited about the HC-1. "Commodity platforms have flattened, and more and more cores have not resulted in more performance. We looked at the problem with a certain amount of déjà vu, back to the days when machines had attached vector processors." And the resulting HC-1 machine gives a performance boost on math routines just like those vectors of days gone by.

In early benchmark tests, the HC-1 running a protein sequencing application (an actual application running at Convey's first customer, the University of California at San Diego) showed a factor of 16 improvement in performance compared to a single two-socket Xeon box running the same code. Performance will vary depending on the application and the implementation of the FPGA personality, of course.

The HC-1 will begin shipping to beta customers in February 2009 and will be in full production in the second quarter of next year. A base HC-1 machine with a quad-core Xeon processor and 4 GB of memory on one side and the FPGA co-processor and 8 GB of memory on the other side costs $32,000. If you do the math, that's roughly 16 times the math oomph for about half the price of 16 reasonably configured two-socket Xeon boxes.

This might not be a tough sell at all, if the programming is as elegant as Toal says it is. ®