Original URL: http://www.theregister.co.uk/2011/07/06/cray_xes_microserver_prototype/

Cray stuffs 46 mobile Intel chips into microserver

Packed, racked, and ruggedized

By Timothy Prickett Morgan

Posted in Servers, 6th July 2011 20:41 GMT

The Custom Engineering unit of supercomputer maker Cray and the bespoke ruggedized, military-grade system maker Extreme Engineering Solutions (X-ES) have joined forces to prototype a super-dense microserver based on Intel's mobile Core i7 processors.

Last week, X-ES announced that it had won the contract to develop and manufacture the server node and backplane for a "data center blade" that packs 46 nodes into a single chassis.

Cray is the one actually selling the box to an unnamed customer, and it was responsible for the chassis, the high-voltage power supply, packaging, and high-velocity air cooling system for the blade server. X-ES designed the server nodes and the backplane.

Neither company is saying much about the compact server, much less giving it a name. But according to a source speaking with The Register on the condition of anonymity, the backplane for the data center blade server measures about 12-inches wide by 26-inches deep.

With the blades installed, it is about four-inches high outside of the chassis. That is just about the right size to fit into a standard 4U chassis with room left over for chassis power distribution.

X-ES Cray microserver

The unnamed 46-node Core i7 microserver from Cray and X-ES (click to enlarge)

The server nodes, which plug into PCI-Express 2.0 peripheral slots, use that PCI bus to power the nodes and to provide links to a Gigabit Ethernet switch that is embedded in the system and that provides interconnectivity for the server nodes. There are two rows of blade servers in the system, and the board is laid out with two rows of 12 blades in the front and two rows of 11 blades in the back.

Some electronics plug into the middle of the board and another set of electronics plug into the rear of the board. The choice of 46 blades is a weird one – why not 48? – but it probably has to with thermal windows and the desire to use some of the PCI-Express slots for the Gigabit Ethernet switches.

Arrandale, not Sandy Bridge

The feeds and speeds of the server nodes are not available, but our source tells us that the boards are based on Intel's "Arrandale" Core i7 processors, not the latest "Sandy Bridge" variants, and that these processors are not put into standard sockets where they can be swapped out but rather directly mounted on the board.

Ditto for the DDR3 main memory, which is also soldered right to the board and which has ECC data protection. By doing this, X-ES can keep a very compact design and better control airflow around the components to cool them.

The Arrandale Core i7 processors date from January 2010 and have two cores plus two DDR3 memory channels. They plug into the BGA-1288 socket and come in standard versions that clock as high as 2.67GHz to 2.8GHz at 35 watts and as low as 1.07GHz to 1.47GHz at 18 watts. Splitting the middle of the road between performance and power consumption would mean using maybe the 2.27GHz low-power variant of the Core i7 chip in the Arrandale family, which came out in September 2010.

The four-core "Sandy Bridge" Core i7 chips for mobile computers are likely too hot at 45 watts and 55 watts, and they don't have any obvious benefits over the Arrandales for two-core versions. The on-package graphics chips in the Sandy Bridge chips are basically useless to server workloads, hence the desire to stick with the mobile Arrandales.

Interwebs love

At four inches high, the 46-node microserver created by Cray and X-ES could easily fit in a 2.5U form factor, cramming 92 cores in that space and maybe 184GB of memory.

We estimate that the microserver chassis would draw 1,300 to 1,400 peak watts of juice if the 25 watt Arrandale parts are used, and would run under normal to heavy load at under 1,000 watts.

Cray is best known for its supercomputers and their petaflops-class number crunching, but this particular 46-node data center server is not designed for such work. Rather, it is designed for a hyperscale web customer.

That customer might be Microsoft, which tapped Cray in March 2010 to research and prototype "a system that could provide a glimpse into the future of cloud computing infrastructure".

This Microsoft contract, which was with the software giant's Extreme Computing Group within Microsoft Research, was Cray's first Custom Engineering contract out of the traditional HPC space.

X-ES was founded in 2002, and the one thing that the company is clear about is that its design center is in Middleton, Wisconsin, and that even though it farms out printed circuit board manufacturing, it does so to a PCB maker located in the United States.

And while the company outsources the surface mounting of components onto boards, it does so with a machine of its own that it put in the contract manufacturer's fab – which is also located in the US of A. Being able to say that means getting those military contracts for embedded systems. ®