Peeling back the skins on IBM's Flex System iron

Flexing an x86 node

As best I can figure, IBM was hoping to launch the Flex System chassis along with Intel's "Sandy Bridge-EP" Xeon E5-2600 processors last fall, in the wake of an expected announcement by Intel – perhaps in September. But Intel pushed out the launch (due to an issue with the on-chipset SAS controller that was supposed to run at 6GB/sec but which was geared down to 3GB/sec before its launch) and that left IBM either leading with Power7 nodes and talking about Xeon nodes, quickly slapping together an Opteron 4200 or 6200 node based on the chips from Advanced Micro Devices, or waiting. IBM chose to wait.

It is not clear if IBM will ever do an Opteron based node, AMD's advantage with the 4200 and 6200 processors over the Xeon E5-2600s (and the forthcoming Xeon-2400 for cheaper and less expandable two-socket machines) is nil when it comes to number crunching and not huge when it comes to supporting large numbers of virtual machines. AMD may have a price advantage, but there may not be enough pull in the market for IBM to invest the engineering effort.

I think there is no question that Big Blue will offer multiple Xeon-based nodes for the Flex System chassis, including a two-bay machine delivering four sockets based on Intel's upcoming Xeon E5-4600 processor, and maybe even a four-bay machine based on a future Xeon E7 processor with four or eight sockets and offering much larger main memory and expansion for the node. I think it is a no-brainer that the Flex System will very soon support Nvidia Tesla GPU co-processors for supercomputer customers, too. A four-bay, eight-socket Power7 node is also a possibility, and with such a configuration, IBM could cover the bulk of its Power Systems customers in either AIX or IBM i customer bases.

In the meantime, the only x86-based server node for the Flex System chassis is the x240, a two socket node based on Intel's Xeon E5-2600 chip. Here's what it looks like from above with the cover off:

The Flex System x240 node is a two-socket box, with is really the only option with the Xeon E5-2600 chips. IBM is offering the full slate of Xeon E5-2600 processors (which I wrote about in detail here) for the machine, ranging from parts with low core counts and low prices to ones with low core counts and higher clock speeds and relatively high thermals to chips with six or eight cores with standard thermals and burning 95, 115, or 130 watts to the top-bin E5-2690 which has eight cores, 16 threads, runs at 2.9GHz, and burns 135 watts. This is probably the thermal zone that the eight-core, 3.55GHz Power7 processor is in, by the way. Which is why they can be crammed into the same space in a Flex System rack with generally the same system features.

The x240 node has 24 memory slots instead of the 16 slots in the companion p260 Power7 node, and thus it can support more memory by default. The fact that Intel is supporting load-reduced (or LR-DIMM) main memory, which allows for memory sticks with up to 32GB of capacity per memory slot, means that this two-socket x240 node can support up to 768GB of main memory. That's three times the main memory of the p260 node. I think that 16GB and 32GB memory sticks are too expensive for most shops at this point, unless they are running heavily virtualized workloads and they want to cram as many VMs or LPARs on a machine as they possibly can. So the memory advantage on the x240 is really only a factor of 50 per cent on machines in the Flex System using 8GB DDR3 sticks. (192GB on the x240 versus 128GB on the p260.)

I also believe that IBM needs to get Power7+ processors into the field soon sporting LR-DIMM memory support and if not on-chip PCI-Express 3.0 support, then at least some way to hang PCI-Express 3.0 slots off the GX++ bus on the Power chips. IBM needs to draw even with Intel on the feeds and speeds and slots and watts.

IBM has two 10 GE ports welded onto the motherboards, which means they don't eat a mezzanine card slot. Rather than putting the 2.5-inch disks under the server node lid, the x240 has two hot-swap disk bays in the front of the bay. IBM is supporting either SAS or SATA disk drives or SSD flash drives in a 2.5-inch form factor for this server node for local storage. SAS drives come in 146GB, 300GB, 500GB, 600GB, 900GB, and 1TB capacities at various rotational speeds; SATA disks come in 250GB, 500GB, 1TB capacities; and SATA SSDs come in 50GB, 128GB, 200GB, and 256GB capacities. There's also a redundant pair of internal USB ports for storing VMware's ESXi hypervisor on baby flash units. The x240 node has the same two mezzanine cards and options as the p260 node. ®