SeaMicro pushes 'Atom smasher' to 768 cores in 10U box

The drumbeat of system upgrades continues apace at microserver startup SeaMicro, with the company launching its third server node for its SM10000 "Atom smasher" in the past nine months.

The new machine is based on the same Atom N570 chip and related NM10 Express chipset from Intel that was used in the prior generation of SeaMicro machines that were launched in February, dubbed the SM10000-64. The difference between that machine and the new SM1000-64HD is that SeaMicro has crammed up to 768 Atom cores in the box, an increase of 50 per cent.

Both of the SeaMicro machines to come out in 2011 are based on the Atom N570, which started shipping in the first quarter of this year, is a dual-core, 64-bit version of the Atom chip that runs at 1.66GHz and that supports HyperThreading. With HyperThreading, the N570 presents four virtual instruction streams to the operating system running on the chip, and on many workloads it can boost performance on the order of 20 per cent or more. The N570 has 1MB of L2 cache on the chip and has an 8.5 watt thermal design point (TDP). This Atom chip may be a 64-bit chip, which is a requirement for a lot of the hyperscale Web data centers that are interested in buying the SM10000 machines, but the N570 does not support ECC error correction on its DDR3 memory controller, which is a bit of a big deal for some server shops, and does not support Intel's VT chip-level virtualization and VT-d I/O virtualization technologies. All of these technologies are supported on Intel's Xeon server chips, of course.

The first generation of SeaMicro machines, the SM10000, came out in June last year and started shipping in volume in September. The SM10000 was based on the 32-bit Atom Z530 processor, and crammed up to eight of these single-core chips on a single server card and then put 64 of these cards into the 10U rack-mounted chassis, for a total of 512 cores.

Back in February, when the SM10000-64 launched, the core count stayed the same at 512 cores, but the architecture shifted to 64-bits. You could see that the server card had some extra space on there, which SeaMicro is using in its latest server board to increase the compute density of the 10U chassis. Here's what the new card looks like:

The server card for the SM10000-64HD

So why didn't SeaMicro just put six Atom N570 processor/NM10 chipset pairs on the server card back in February? Andrew Feldman, CEO and co-founder at SeaMicro, tells The Register that the answer was "engineering expediency".

The problem is that the load balancer and torus interconnect at the heart of the SeaMicro system was tuned for 512 cores. And the easiest and quickest way to get 64-bit support out the door was to keep the core count the same and then spend the spring and early summer tweaking the SM10000's various microcode to support more cores. That is what has happened with the SM10000-64HD.

Tweaking this interconnect is not a trivial thing. The SeaMicro design includes a home-grown ASIC chip that virtualizes disk access and Ethernet networking for each of the Atom servers. The ASIC also implements a multi-dimensional torus interconnect between all of the server nodes, which is similar to the interconnect that IBM developed for its BlueGene massively parallel Linux supercomputer and which delivers 1.28Tb/sec of aggregate bandwidth across the 64 server motherboards.

SeaMicro also cooked up a field programmable gate array (FPGA) to do load balancing, and these circuits are hooked into the SM10000's system management tools to allow for pools of servers to be grouped together and managed as a single object and to provide guaranteed performance levels for groups of processors, disk, memory, and fabric. The fabric doesn't have a name, but this function performed by the fabric is called Dynamic Compute Allocation Technology, or DCAT.

With the ASIC interconnect, load balancing, and DCAT features all tuned up, now SeaMicro can put the full complement of six processors on the latest server card. Each socket gets 4GB of DDR3 memory, for a total of 1.5TB per chassis (that's 2GB per core).

The chassis supports up to 64 SATA disk or solid state drives and Gigabit Ethernet (64) or 10 Gigabit Ethernet (16) uplinks out of the back of the box to reach the outside world. The server cards plug into the SM10000 chassis and backplane through two PCI-Express 2.0 slots. The backplane currently supports 128 of these PCI slots for hooking in server boards. (Two columns of 32 slots on each side of the chassis.)

Just by increasing the processor count by 50 per cent on each server card, SeaMicro is able to decrease power consumption per processor by 18 per cent because the ASICs and interconnect are shared across more computing units without any extra power draw.

The SM10000, SM10000-64, and SM10000-64HD servers all look the same from the outside.

An original SM10000 server with 512 cores and 1TB of main memory cost $139,000. The bump up to the 64-bit Atom N570 for 512 cores and the same 1TB of memory boosted the price to $165,000. A 768-core, 1.5TB machine using the new 64HD cards will run you $237,000. That's 50 per cent more oomph and memory for 43.6 per cent more money. ®