Original URL: http://www.theregister.co.uk/2012/03/27/amd_seamicro_server_plans/

AMD plots an end run round Intel with SeaMicro's 'Freedom'

Can clouds and virt move the battle off the chip?

By Timothy Prickett Morgan

Posted in Virtualization, 27th March 2012 13:18 GMT

Analysis As last week was winding down, Advanced Micro Devices took control of upstart server maker SeaMicro, and guess what? AMD is still not getting into the box building business, even if it does support SeaMicro's customers for the foreseeable future out of necessity.

Further: Even if AMD doesn't have aspirations to build boxes, the company may be poised to shake up the server racket as a component supplier. Perhaps not as dramatically as it did with the launch of the Opteron chips nearly a decade ago, but then again, maybe as much or more - depending on how AMD plays it and Intel and other server processor makers react.

AMD announced its intent to acquire SeaMicro back on February 29, and the deal was consummated on Friday (March 23) with $293m in cash and AMD's assumption of nearly 6.5 million shares of its own stock and the issuance of another 322,000 shares of restricted AMD stock. All told, AMD shelled out $334m to get its hands on SeaMicro, and perhaps more importantly, to get its hands on the 3D torus interconnect and other electronic features at the heart of the SM10000 family of servers.

The SeaMicro chassis can hold 64 processor cards – 32 up one side and 32 down the other – as well as a slew of disks and network ports to the outside world. The genius of the design is not that it crams a bunch of Atom or Xeon servers on these baby PC server cards, which hook into the system backplane through PCI-Express x16 slots, but rather all of the sophisticated gadgetry that links the servers together and allows for each server card to have nothing but processor sockets, chipsets, and memory slots.

AMD is already feeling quite proprietary about the SeaMicro server interconnect, which was called "Freedom" internally at SeaMicro and which will no doubt get a properly boring name if the AMD marketeers have anything to day about it. (Opticonnect is already owned by IBM, so forget that one, but Opterlink or HyperNetwork might be interesting possibilities.)

"Our unique fabric technology is truly one of the crown jewels of the cloud," said Lisa Su, senior vice president and general manager of Global Business Units at AMD, in a statement. "The combination of this innovative technology with our processor design expertise greatly enhances our ability to attack the fastest growing portion of the server market with industry-leading low-power, low-cost, high-bandwidth solutions."

AMD Freedom interconnect

AMD's "Freedom" interconnect

The Freedom interconnect has 1.28Tb/sec of aggregate bandwidth on that 3D torus, the same as the prior generation. It supports up to 16 external 10 Gigabit Ethernet links or 64 external Gigabit Ethernet links to the world outside of the 10U chassis in the SM10000 machine. The chassis can currently support 64 quad-core Xeon processors from Intel or 384 dual-core Atom processors from Intel; in both cases, the SeaMicro designs use single-socket processors only.

That torus interconnect doesn't just link the server nodes to the disk and networking I/O in the chassis. It also virtualizes access to I/O in the system so you can reconfigure capacity allocations of that 1.28Tb/sec of bandwidth to different nodes inside the system. The system also has a field programmable gate array (FPGA) to do load balancing across all server nodes, and these circuits are hooked into the SM10000's system management tools to allow for pools of servers to be grouped together and managed as a single object and to provide guaranteed performance levels for groups of processors, disk, memory, and fabric. This feature is called Dynamic Compute Allocation Technology, or DCAT. The SeaMicro chassis supports up to 64 SATA disk or solid state drives.

With the Freedom interconnect, SeaMicro added an interesting feature called TIO, which is short for Turn It Off. SeaMicro has hacked functions into the interconnect that can reach into the Intel C200 chipset and the Xeon E3 chip and turn off all the stuff that a SeaMicro server node does not need. With the Freedom interconnect virtualizing the disk I/O, the C200 chipset doesn't need to power up its SATA interface or its USB ports; it also tells the HD 2000 graphics controller on the Xeon E3 to shut up and shut down.

Why the SeaMicro play?

If it is not completely obvious by now, AMD and its foundry partners, GlobalFoundries and Taiwan Semiconductor Manufacturing Corp, are never going to catch up to Intel in the chip process arms race. Intel has anywhere from a full node to two full nodes of process lead on these companies, depending on the when you look at it over the past several years and looking ahead a few years into the future. AMD's new management correctly sees that it cannot win a chip process war.

Look at what Intel does. With every process shrink, it can cram more stuff onto the chip. A long time ago – it seems like ancient history now – we were happy with a single core on a die, and then we started integrating various levels of cache memory and cache controllers onto the chips to goose performance. Then came multiple cores and main memory controllers, virtualization circuits, and in the latest Xeon E5-2600 processors, there is a very sophisticated ring interconnect that lashes cores, cache memories, QuickPath Interconnect for gluing together multiple processors, and the whole I/O subsystem (including PCI-Express controllers) onto the die. How long before there will be integrated Ethernet and InfiniBand ports on the Xeon chips – or better still, a single network interface controller than can speak either?

Maybe, just maybe, you externalize and virtualize these things that Intel is cramming onto one chip

So what is the number two server processor chip maker to do? Try harder, and change the conditions of the test. If you have a process lag, you stop trying to cram everything onto the chip and maybe, just maybe, you externalize and virtualize these things that Intel is cramming onto one chip. That way, you can use whatever node you are at process-wise to make the CPUs or APUs and make that processor-agnostic Freedom interconnect and its FPGA software what amounts to a combined server chipset and interconnect. You do two big chips of roughly equivalent size to Intel's two chips of wildly divergent size. (The Xeon is big, the chipset is getting smaller and smaller.) As long as in the aggregate system makers can build a lower-cost, high-performing, better-integrated, power-sipping system out of the components, they won't much care how big the respective chips from Intel and AMD are.

The meteoric rise of SeaMicro, which has been growing faster than many infrastructure startups from the dot-com era did and which makes it well worth the $334m in cash that AMD forked over, demonstrates this – provided AMD uses the technology intelligently and makes no promises it cannot keep.

AMD Opteron server roadmaps

AMD's possible chips for SeaMicro machines (click to enlarge)

For now, the big problem that AMD faces is getting its own processors working inside of the SM10000 machines. The most obvious processor for AMD to park on a SeaMicro system card is the new Opteron 3200, launched last week. This chip is aimed precisely at the same microserver customers that Intel is chasing with the Xeon E3-1200 processors. With prices ranging from $99 to $229 for variants with four or eight cores, AMD can make a credible argument in terms of performance and price/performance compared to the Xeon E3s with two or four cores and slightly higher prices. The Opteron 3200s plug into the AM3+ socket and don't have as many HyperTransport links or as much bandwidth on the point-to-point interconnects as other Opteron 4200 and 6200 processors: so maybe AMD will go with the Opteron 4200s in the SM10000s when it delivers its own processors in server nodes sometime in the second half of the year. Even big fat Opteron 6200 nodes are not impossible, since you could - if you wanted - use these 16-core behemoths in a single-socket server node. (Super Micro sells such a board.)

Michael Detwiler, product marketing manager at AMD, was not being coy with El Reg when he would not disclose what Opteron processors AMD would eventually slide into the SeaMicro chassis. "What we're going to do is talk to customers and bring to market what makes sense for them."

AMD SeaMicro server market

AMD targets cloudy servers with SeaMicro buy (click to enlarge)

It stands to reason that there will therefore be more than one option, just as SeaMicro has used a variety of Intel chips thus far in its machines. And the reason is that no one single node is going to appease the cloudy server buyers that AMD is trying to target with both the Opteron 3200 and 4200 processors and now the SeaMicro interconnect.

It would be foolish to think that AMD is only thinking about cloudy servers as it ponders the possibilities of the SeaMicro acquisition. For one thing, the Freedom interconnect can, in theory, scale to thousands of server nodes and if you plunked an Opteron 6200 processor on a system board and put a FireStream GPU next to it on the board, you could make a very powerful ceepie-geepie server node that, with the 3D torus interconnect, would make a pretty respectable supercomputer by any current definition of the thing.

This is particularly important since Cray has made it pretty clear that its next-generation "Cascade" systems and "Aries" interconnect will not be reliant on AMD's Opteron processors and HyperTransport point-to-point interconnect between the Opterons and the Aries chips (which also implement a torus). With AMD essentially losing its biggest OEM partner in the HPC space, AMD is very likely going to help other HPC aspirants go after Cray with the Freedom interconnect and cheap Opteron processors. That's what El Reg would do in this situation.

The SeaMicro ASICs could also be used as out-of-band appliance for virtualizing I/O for infrastructure aimed at supporting more enterprise-grade server virtualization workloads, which are distinct from cloudy workloads in that their resilience is in the virtualization layer, not embodied in the software stack itself.

Cloud, virtualization, and HPC are the growth markets, and it stands to reason that AMD will take all of the components of the SeaMicro system and rejig them so they can be sold as components that AMD's server OEM customers can use to build their own systems.

For more general purpose workloads, AMD could even go so far as to create fatter SMP nodes that span multiple server boards (much as Sun Microsystems did with its very clever uniboard designs for Sparc and Opteron systems) and maybe even implement the chipset in software atop the Freedom interconnect. Imagine what would happen, for instance, if AMD snapped up ScaleMP and created a software-based chipset that lived in the SeaMicro ASIC and that could create virtual SMPs in a chassis or across enclosures on the fly.

The possibilities are not endless, but they are intriguing. ®