IBM sneak peeks Nehalem EX iron

The counterpunch

The eX5 chipset has a counterpunch to the memory expansion capability of selected models of the "California" Unified Computing System B-Series blade servers from Cisco and their C-Series rack variants. Cisco's own memory expansion ASIC works in conjunction with the on-chip memory controllers of the Xeon 5500 processors and the related Intel chipsets to allow a rack or blade to support a maximum of 384 GB instead of the 144 GB limit set by the Intel electronics. (That's a factor of 2.7 times the memory expansion).

IBM has come up with its own memory expansion technology for the Nehalem-EX boxes and the eX5 chipset, called Max5. With this feature of the chipset, an add-on chassis that just has DDR3 memory DIMMs can be linked to a rack server. Ora similar blade form factor with extra memory slots can be linked to a blade server in much the same way that multiple nodes in high-end System x and Power systems can be lashed together to make SMPs. These memory expansion units are basically server motherboards with no server sockets, and the memory is available over fast QPI links to the servers they are linked to.

IBM is planning to launch two-socket and four-socket Nehalem-EX rack servers and a two-socket blade. The names, feeds, and speeds of these machines were not divulged. But Bradicich did say that for the rack servers, the Max5 memory expansion feature would come in a 1U box with 32 DIMM slots and the Max5 blade expansion feature would have 24 DIMM slots. The two-socket rack machine will have 32 native memory slots, the two socket blade server will have 16 native slots, and the four-socket socket rack machine will have 64 slots.

So you are talking about 256 GB of relatively cheap memory on a two socket blade server (using 4 GB DIMMs that cost maybe $250) and even 512 GB for somewhat pricey 8 GB DIMMs (which sell for about $1,000). Of course, heaven only knows what IBM will want to charge for the Max5 features and the cable to link them into the SMP cluster. As for capacity, IBM can put 768 GB into a four-socket box using 8 GB DIMMs, and 1.5 TB if customers want to go crazy with 16 GB DIMMs. "We think this is more than enough for this machine," Bradicich says with a laugh.

The eX5 chipset and server design is also more granular than the prior eX4 products. With these, IBM had Xeon 7400 processors and four sockets and their main memory were the basic building blocks of a machine that scaled up to 16 sockets. With the eX5, IBM is using a two-socket or four-socket mobo as the basic building block.

For now, all IBM says for sure is that the chipset will allow for two two-socket rack or blade servers that are matched in terms of processors and form factors to be linked together as an SMP using what it calls a FlexNode feature. IBM has been offering two-way to four-way upgrades in its LS23/LS43 Opteron-based blade servers (using the HyperTransport bus) for two years now. This is similar, but SMP is not just limited to blades. IBM is also allowing two four-socket rack servers to be linked together into an eight-way.

Initially, it looks like the FlexNode feature is just supported with two machines, and only one Max5 memory expansion unit can be plugged into a FlexNode setup. The interesting bit is that IBM's Systems Director software will apparently allow for a FlexNode configuration to be merged and broken on the fly, as workloads dictate. IBM's System x box was the star of the Nehalem-EX preview from Intel last May.

As for the scalability for processor sockets and memory in the eX5 chipset, Bradicich would not say anything about what larger Nehalem-EX machines Big Blue is cooking up. But the architecture almost certainly scales to 16 sockets, as the previous ones did, and maybe a bit further unless IBM is afraid of compete with its Power7 big iron due later this year, which will scale to 32 sockets using those eight-core Power7 chips. What Bradicich would confirm is that the eX5 chipset would not support Advanced Micro Devices' Opteron processors and did not want to talk about IBM's plans for supporting the future "Magny-Cours" 12-core Opteron 6100s, also due out this month. ®