Dell ARMs up for hyperscale servers

But are they dangerous to anyone except HP?

Build a business case: developing custom apps

Look out Intel. Here comes another ARM box to the microserver party.

If people didn't want ARM-based servers, Dell wouldn't build them, and so with the launch of the "Copper" ARM server sled for the "Viking" C5000 microserver chassis we know that people want ARM servers. And this is not some experiment that Dell is doing, either, Steve Cumings, executive director of marketing for Dell's Data Center Solutions bespoke server unit, tells El Reg.

"This is not only real, but it is going out the door," says Cumings, with a jab at rival Hewlett-Packard's "Project Moonshot" hyperscale server effort and its "Redstone" many-ARMed server nodes launched last November. Those servers made a big splash, but neither HP nor Calxeda have said much since then about who is using these concept machines – or what the plan is to ramp up the Redstone product line with other ARM chips with more oomph and memory capacity.

Those HP Redstone servers are based on Calxeda's 32-bit ECX-1000 processors, which also debuted last fall, and put four processors on a card, 18 cards in a tray, and 288 nodes in a 3U space – all interconnected by the EnergyCore Fabric Switch embedded on each Calxeda chip. This interconnect can implement a 2D torus, mesh, fat tree, or butterfly tree topology and can scale across 4,096 sockets. This is just the thing for certain hyperscale web and data-munching workloads that need lots of compute cores but not the high-bandwidth interconnect that top-end Ethernet or InfiniBand switches and adapters would offer.

Cumings says that the Copper server sleds have been in development and test for the past year among its hyperscale server customers, who generally are looking for the cheapest clocks and lowest watts to cram more computing per dollar and per watt into their server racks to support various workloads.

The reason why it has taken Dell so long to get a production-grade ARM server to market has little to do with hardware – 32-bit and 40-bit ARM chips that could be used in servers have been around for a while – but rather is related to the fact that the ecosystem of software had to evolve around the machines to give them something useful to do. Even the most hyper of the hyperscale data centers does not want to roll its own Linux and software stacks unless it absolutely has to.

Dell's ARMed PowerEdge C5000 microserver chassis

Dell's ARMed PowerEdge C5000 microserver chassis

Now that Ubuntu 12.04 LTS is out with an ARM variant, there is a commercial-grade Linux and application stack you can run on an ARMv7-compatible chip. And the Fedora Project is also doing tweaks to support ARM chips, allowing those who like RHELish Linux to experiment as well.

The LAMP stack and Hadoop will run on these Linuxes on ARM processors, and the OpenStack cloud controller was demoed running on LARM (what else are we going to call it?) this month, too. The KVM hypervisor is expected to be ready soon for the Cortex-A15 ARM processor and Java also works on the chip, too. Dell is also working to get its "Crowbar" server configuration tool working on the Copper servers before the end of the year, and is working to get a variant of Crowbar that can set up Hadoop clusters out the door, too.

The important thing is not that all of this software is running or will soon be running, but that the software is getting architecture-specific optimizations as people see the hardware and software ecosystem for ARM-based servers coming into being.

"We spend a lot of time on hyperscale, and we have a pretty clear understanding of what customers want to do and where their interests lie," explains Cumings. "We have had this system in the lab for over a year, and we have been trying to figure out what it might be good for."

Initially, it looks like what you might expect: Web front ends and Hadoop data munching are the two big workloads that will deliver better performance per watt and better performance per dollar compared to X86 servers.

Cumings is not about to provide any specifics about how much better or what these things cost, but each ARM server node on the Copper sled only burns a peak of 15 watts. That's pretty low, and as low as Intel is trying to shoot with its Xeon and Atom chips.

Dell has chosen Marvell's Armada XP 78460 variant as the basis of the Copper sled. This is a quad-core system-on-chip (SoC) with four Marvell "Sheeva" PJ4B cores, which are a variant of the ARMv7-MP design that has the 40-bit memory addressing added to it. The cores run at 1.6GHz and have both symmetric multiprocessing and asymmetric multiprocessing enabled. SMP allows the cores to be ganged up into a single image behind main and cache memories, while AMP allows the cores to be carved up into isolated virtual hardware slices.

Block diagram of Marvell's Armada XP 78460 chip

Block diagram of Marvell's Armada XP 78460 chip

As you can see, the heart of the Armada XP 78460 is a system crossbar interconnect that links all of the elements of the SoC together – components that might otherwise be on a server motherboard separated from the CPUs and glued together by a chipset. Each Sheeva CPU has a floating point unit and is linked over a coherency fabric to a 2MB on-die L2 cache. The memory controller supports up to 8GB of DDR3 main memory with ECC scrubbing and running at up to 1.6GHz. There are four PCI-Express 2.0 controllers on the chip, as well as controllers to drive two SATA peripheral ports and four Gigabit Ethernet ports. There is also a 4Gb/sec packet processor that could come in very handy, as well as a security engine for encrypting and decrypting data and a controller to link to three USB 3.0 ports. That is a lot of stuff to cram into a 15 watt thermal envelope – hence the excitement about ARM servers.

Dell Copper ARM server sled

Dell's "Copper" ARM server sled (click to enlarge)

The Copper sled server, which slides into the PowerEdge C5000 chassis from the DCS unit, puts four of these Armada XP 78460 processors on a system board with four memory slots; each processor links to one slot and it has 8GB of memory in it. This board is designed by Dell; it is not clear where it is manufactured. The sled has room for four 3.5-inch SATA drives, one for each ARM server node.

Off to the right of the four processors is another processor and some other gadgetry that implements a Layer 2 Ethernet switch across all of the four nodes on the Copper sled as well as the dozen Copper sleds that can be slid into the C5000 chassis. This network can also span multiple enclosures if customers want to do it. The exact feeds and speeds of this on-board, distributed Ethernet switch are not being divulged at this time, but it was clearly designed to compete against the EnergyCore Fabric Switch embedded on each Calxeda ARM server chip. Cumings says that Dell expects for customers to use normal top-of-rack switches to link multiple Copper sled enclosures together, but they don't have to.

Cumings says that the Copper sleds have been shipping to selected seed customers for some time and that Dell will be standing up racks of the ARM-based servers in its solution centers as well as in the Texas Advanced Computing Center (TACC) at the University of Texas. The latter is a supercomputing center that has the facilities to allow secure remote access from all over the world, and Dell's plan is to let developers sign up for some timeslices on the Copper servers to give their code a whirl before they commit to buying the custom iron from DCS.

A single C5000 chassis can hold 48 ARM processors, for a total of 192 cores. That works out to 2,688 cores in a rack if you fill it top to bottom with C5000s – or 2,496 cores if you leave 3U open for top-of-rack Ethernet switches.

The HP Redstone machines can cram a lot more cores into the rack if you don't put storage on them, and still offer higher density if you put SSDs or 2.5-inch SATA disks into some of the SL6500 enclosure's trays. The issues then become the cost of each machine and the premium, if any, that HP is commanding for the Redstone density and the Calxeda interconnect. As Copper and Redstone are not shipping commercially and do not have official prices, it is hard to tell how it all plays out. ®

The essential guide to IT transformation

More from The Register

next story
Reg man looks through a Glass, darkly: Google's toy ploy or killer tech specs?
Tip: Put the shades on and you'll look less of a spanner
So, Apple won't sell cheap kit? Prepare the iOS garden wall WRECKING BALL
It can throw the low cost race if it looks to the cloud
One step closer to ROBOT BUTLERS: Dyson flashes vid of VACUUM SUCKER bot
Latest cleaner available for world+dog in September
Samsung Gear S: Quick, LAUNCH IT – before Apple straps on iWatch
Full specs for wrist-mounted device here ... but who'll buy it?
Apple promises to lift Curse of the Drained iPhone 5 Battery
Have you tried turning it off and...? Never mind, here's a replacement
Now that's FIRE WIRE: HP recalls 6 MILLION burn-risk laptop cables
Right in the middle of Burning Mains Man week
Apple's iWatch? They cannae do it ... they don't have the POWER
Analyst predicts fanbois will have to wait until next year
Noo! Hand not big enough! Don't look at meee!
AMD unveils 'single purpose' graphics card for PC gamers and NO ONE else
Chip maker claims the Radeon R9 285 is 'best in its class'
prev story


Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Backing up distributed data
Eliminating the redundant use of bandwidth and storage capacity and application consolidation in the modern data center.
The essential guide to IT transformation
ServiceNow discusses three IT transformations that can help CIOs automate IT services to transform IT and the enterprise
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.