Original URL: http://www.theregister.co.uk/2011/05/02/hp_tukwila_big_iron/

HP forges 32-socket Itanium iron

Beyond Superdome

By Timothy Prickett Morgan

Posted in Servers, 2nd May 2011 03:00 GMT

What's going on with Hewlett-Packard's Itanium-based Integrity server lineup? It has been over a year since HP got its first machines using Intel's "Tukwila" quad-core Itanium 9300 processors into Integrity machines, and the company has still not delivered machines spanning 32 and 64 sockets. This is the kind of big iron that key HP-UX accounts need.

IBM got its big, bad Power 795 AIX box out the door last fall, sporting up to 256 Power7 cores in a single system image, and the machine was one of the key factors driving up the company's Unix server sales by 19 per cent in the first quarter of this year. Oracle and Fujitsu are in super-slow-mode with the Sparc Enterprise M9000, sticking with the existing quad-core Sparc64-VII+ chips and able to field 256 cores in a 64-socket configuration. The Superdome 2 is stuck at 64 cores.

The lack of a high-end Integrity server running HP-UX is simultaneously a boon for Oracle, which wants to sell its own (or rather, rebranded Fujitsu) Sparc Enterprise M8000 and M9000 servers, as well as a bust because Oracle does in fact sell a lot of its database, middleware, and application software on HP-UX machines.

If HP can't deliver big Superdome 2 boxes, Oracle might as well bash the Itanium chip, as it has been doing. Oracle says that Intel and HP are not being honest about the future of Itanium, that Intel is focused on Xeon processors, and has therefore said that it will not develop future versions of its database, middleware, and application software for the Itanium platform. And when Oracle says that, what it is really saying is that it is stopping development on HP-UX.

Publicly, Intel has stuck to its party line, saying that it has two more Itanium generations coming down the pike, but has told its Itanium customers privately that it is considering its options for what happens after the "Kittson" Itaniums come to market in 2014 or so. The eight-core "Poulson" Itaniums, now confirmed for 2012, will plug into the current Integrity blade and rack servers, which were announced last April, and they are also plug-compatible with the Superdome 2 blade-ish SMP boxes, which currently come with 8 or 16 sockets in a single system image using the Tukwila Itanium processors.

The sx3000 chipset implements a crossbar interconnect that provides up to 32 links per blade enclosure; each enclosure can hold a maximum of eight blades. Kirk Bresniker, vice president and chief technologist for the Business Critical Systems division at HP (and an HP Fellow as well), tells El Reg that this Superdome 2 crossbar is based on HP's own electrical design and uses its own communication protocols, which have their heritage in the Convex supercomputers and the PA-RISC and Itanium Superdome machines.

The early Superdomes had their own custom cables to implement the interconnect, but with the SD2 machines, HP has switched to PCI-Express cabling, which saves money. In any event, these crossbar links can be used to lash multiple processor boards together and to link out to external I/O expansion enclosures. That crossbar is implemented in separate blades that go in a separate chassis on top of the enclosure for the processor blades, linking out to external I/O or adjacent crossbar blades feeding into other Superdome 2 blade enclosures.

The main differences between the SD2-8s and SD2-16s machines, as these servers are called, is the size of the nPar partitions that run on the boxes and external I/O expansion. Both are based on two-socket extended blades that have the sx3000 chipset from HP plugging into the QuickPath Interconnect (QPI) point-to-point links on the Tukwila chips to implement SMP clustering for server nodes.

These machines support the Itanium 9340 and 9350 processors, from 32GB to 2TB of DDR3 main memory, 24 PCI-Express 2.0 slots, and 32 10 Gigabit Ethernet ports. The SD2-8s has nPar hard partitions that can span as large as eight processors and 32 cores, has up to 48 external I/O ports that can be attached into the sx3000 chipset's crossbar. The SD2-16s can have an nPar hard partition that can extend across 16 sockets and 64 cores (that's the entire box) and has double the external PCI-Express peripheral slots, at 96.

The HP Integrity server roadmap from late 2009 had the Integrity blade servers - the two-socket BL860c i2, four-socket BL870c i2, and the eight-socket BL890c i2 - being launched in April 2010, with the SD2-16s coming in August 2010.

HP Tukwila Roadmap Circa Late 2009

HP's Tukwila roadmap, circa late 2009 (click to enlarge)

There was no mention of an SD2-8s in that late 2009 roadmap, but the SD2-32s, a 32-socket server sporting 128 cores and spanning two SD2 chassis, was due in the first half of 2011 and the rx2800 i2 rack server - mostly aimed at OpenVMS customers but also sporting Windows Server 2008 R2 - was due in the second half of 2010. As we know, HP kicked out the Integrity BL8X0c i2 blades, the rx2800 i2 rack, the SD2-8s, and SD2-16s machines in April 2010 and hasn't touched the product line since.

There have been some enhancements to the NonStop machines, which are also based on the Integrity platform, and at some point last year, HP tweaked the Superdome 2 online specs to let customers know the SD2-32s machine was coming and that the architecture could scale to 64 sockets and "from 8 to 256 cores and more".

Superdome silence

HP has been pretty vague about when to expect the SD2-32s machine, and it doesn't want to talk about the SD2-64s box at all. "We expect to start shipping in the summer of this year," Bresniker tells El Reg, referring to the 32-socket Superdome 2 machine.

Bresniker was not at liberty to talk about the SD2-64s top-end Superdome 2 machine, but Lorraine Bartlett, vice president of worldwide marketing, strategy, and operations for the BCS unit at HP, said that the release of the Poulson Itaniums in 2012 would be "an interesting data point". That implies, but does not confirm, that HP is waiting until the Poulsons are out to ship the biggest machine, which would allow it to field a 512-core machine using the eight-core Poulson chips. Or to get 256 cores into the field with an SD2-32s. Go figure.

No matter what HP does and does not say publicly, it is showing customers that a 64-socket box is in the works. Such as in this description of the Superdome 2 machines:

HP Superdome 2 Systems

HP's Superdome 2 systems lineup (click to enlarge)

The SD2-64s machine uses the sx3000 chipset and the five QPI links in the Tukwila and Poulson Itaniums to lash four chassis of eight-blade server nodes together to make a single system image with up to 8TB of main memory using 8GB DDR3 main memory sticks. Some of the QPI is eaten up linking the extra CPUs together, so the SD2-32s and SD2-64s machine top out at 96 external I/O slots, which is what the SD2-16s has. The big Superdome 2 machine has 192 PCI-Express slots and 128 10 GE ports, which is a lot of I/O connectivity.

That said, the Power 795 from IBM has 640 PCI-Express slots, and the M9000 from Oracle and Fujitsu has 288 slots. HP's Superdome 2 machines are heavy on the integrated 10GE networking and lighter on the peripheral slots, which is probably sensible in a converged world for networking for servers and storage.

One more thing that may not be obvious. While HP talks about how the Integrity product line supports its HP-UX as well as its OpenVMS and NonStop operating systems, the latter only run on specific machines. You cannot slap OpenVMS 8.4, the most current release, on just any Integrity machine, and in fact, it is only supported on the BL8X0 i2 blades.

Bresniker tells El Reg that OpenVMS 8.4 will span across all of the four blades in the most recent Tukwila blade servers, delivering 32-core scalability, and that customers who need more scalability than that can do VAXclusters lashing multiple blade boxes together. Clearly, OpenVMS does not know how to speak to the sx3000 chipset and hence cannot run on the Superdome 2 machine.

"Most OpenVMS customers do not need the scalability of a Superdome 2," says Bresniker. "Having the scalability to eight sockets on the blades is sufficient for their needs."

The same thing holds true of most of IBM's customers using the OS/400 (now known as IBM i) operating system. The vast majority of customers using this proprietary platform run on two-socket or four-socket boxes, with a small minority needing oomph beyond that. ®