Original URL: http://www.theregister.co.uk/2013/12/16/google_intel_arm_analysis/

That Google ARM love-in: They want it for their own s*** and they don't want Bing having it

Custom RAM chippery, low power, and more

By Jack Clark

Posted in Servers, 16th December 2013 11:01 GMT

Analysis Google, Facebook, and likely other major tech firms are investigating ARM-compatible chips to drive low-power servers, which could ultimately shakeup the types of processors that sit inside data centres.

We were moved to consider the pros and cons of moving away from trusty x86 into the problem-filled world of ARM after a Bloomberg rumor circulated on Thursday that Google was planning to use ARM chips for servers, and after we spotted a smoking gun blogpost by Facebook that indicated it hoped to evaluate ARM systems for production use.

Companies are interested in ARM chips due their historically thrifty power consumption; right from the start, the processor architecture – born from a small team at Acorn Computers in the 1980s – was designed to be RISC (Reduced Instruction Set Computing). This means ARM cores do lots of simple operations relatively quickly; this simplicity, and the lack of legacy cruft to support, keeps the transistor count low, which means less power is required.

This customisable architecture is wildly popular in the battery-powered gadget and embedded electronics worlds, where processing performance isn't key – anything taxing can be offloaded to dedicated silicon – so the chips can be run slower and thus consume even less power.

Compare this to Intel's CISC (Complex instruction set computing) design, which offers a larger range of operations, has a mountain of legacy tech to support – from 16-bit real mode all the way up to 64-bit protected long mode – and generally runs at higher speeds to give punters the most bang for their bucks. All this adds up to beefier, power guzzling packages.

Though ARM cultists portray RISC as being fundamentally better at low-power computation, academic studies have disproved this [PDF], noting instead that the main differences in power consumption come from historical implementations – ARM has spent nearly two decades living in your pocket, whereas Intel has resided in your honking big desktop, and so on. Indeed, the heart beating in today's x86 chips is RISC in design albeit wrapped in a CISC compatibility blanket.

ARM has for a long time focused on cutting power consumption due to its home markets being mobile and non-performance-demanding devices, whereas Intel previously emphasized speed; chips powered by ARM cores are built from the ground up to sip rather than suck current. The drawback is that they beaver away at a relatively leisurely pace.

Mobes and fondleslabs, ARM. Gaming rigs, x86. Got it. Where does Google and Facebook fit in?

Consumer-serving web giants spend billions a year on infrastructure and millions on electricity bills. For these companies, data centres are a great sucking chest wound in their annual budget, and if you can save money, you will.

A key question here is how much of a company's given workload can be shifted onto minimalist, low-power and slower processors. For Google, its compute-intensive backend needs the sheer might of Intel's powerful Xeon chips, but for other tasks such as front-end web serving, and the controlling of storage, it may make sense to deploy ARM.

Facebook is a simpler proposition – its newly minted AI group aside – as the social network's workload is less CPU-intensive than Google's. But even here there are problems: "What we would like to see in an ARM server CPU is a ... minimum frequency of 2.5GHz," Facebook tech strategist Vijay Rao said in late October. "That gives us the ability to scale across our software from frontend to backend."

ARM server startup Calxeda's most recent 32-bit Midway chips had a clock rate of between 1.1 and 1.8GHz, so there is a way to go here. (Yes, there's more to performance than clock rates, but as a general rule of thumb, a part's GHz rating will do for the moment.)

Meanwhile, 64-bit ARM servers are due to come to market next year, with the first likely due from startup Applied Micro, at which point companies will be able to benchmark and rank the tech.

Facebook is known to want to buy community-designed computers – its hardware supply kingpin Frank Frankovsky just joined Calxeda's board – while Google may decide to go ahead and create its own custom kit, including a processor licensed from ARM.

Don't forget that ARM's business model is all about selling copies of its blueprints to the likes of AMD, Samsung, Broadcom and Calxeda, which manufacture customised chips to suit their individual needs and products. Thus, Google will need a partner to fab its chips.

Before that, Google would have to employ about 50 engineers and spend a year or so drawing up its chip, which could be based on the 64-bit Cortex-A57, then another year integrating it with its tech, suggested Calxeda's marketing chief Karl Freund, and would likely need to spend tens of millions of dollars. For Google, which minted $2.97bn in net income in its most recent financial quarter, this is a trivial expense.

FBcoldstoragearray

Just some servers chilling out ... Facebook's 'cold storage' arrays could soon be sitting alongside ARM-powered kit

The next step would be porting Google's software over to ARM systems. The ad giant – a one hundred percent Linux shop, at least in the data centre – has worked hard to create a relatively small, monolithic software stack: assuming it isn't using a Facebook-style x86-64 translator for its code, testing and deploying on ARM will be straightforward for Googlers, although there are all sorts of optimisations that should be explored.

The company has also invested heavily in developing the expertise required to build its own processors. In 2010 it acquired a stealthy chip designer named Agnilux run by refugees from the Apple-bought PA Semi. At the time, the company was thought to be working on either ARM or PowerPC server chips.

'Google wants to invent it, wants parts that are optimized for their own s**t and they don't want Bing to have it'

In the past few years Google has also quietly acquired a number of DRAM and other memory patents associated with a (now out of business) startup named MetaRAM.

The company also employs a number of hardware experts for its data centre gear, and likely for a number of skunkworks projects in its "Google X" advanced products wing.

So - Google has an option and the apparent expertise to do some interesting stuff. What could it get out of ARM that it couldn't get out of x86?

"Google wants to invent it and wants to own it and want parts that are optimized for their own shit and they don't want [Microsoft's] Bing to have it," one highly placed source within the semiconductor industry told The Register.

"Google could be including blocks of intellectual property that advantages their software. They could be putting huge amounts of IO on the part," our contact added, speaking on condition of anonymity.

"They could, since they purchased the patents from MetaRAM, be making modifications in the DRAM controller. I think what we know for sure is that if you control the hardware and software and a server system you have all the levers."

The well-placed source strongly believed it would make sense for Google to at some point do a custom chip as it would give it greater secrecy, greater control over its infrastructure and perhaps the ability to increase the capabilities of its systems by implementing workloads in hardware.

But Intel has been in this game for a long, long time, and as a consequence can bring process advantages and expertise to bear that mean the chance of Google being able to actually develop a better general-purpose chip than Intel is slim.

Intel is going to be taping out 14nm low-power chips next year, which will combine excellent performance with a lower-than-usual power draw. ARM processors, by comparison, will be pumped out of fabs operated by TSMC, Global Foundries, and Samsung, among others, which are thought to be running at the high-20nm node at the moment, and may move to 20nm by end of 2014.

"With over 50 server, storage and communications designs based on 22nm Intel Atom C2000 (Avoton) SoC launched in September, we are well on our way in leading the low-power 64-bit system-on-chips (SoCs) segment," an Intel spokesman told El Reg. "Today, Intel Atom is still the only available 64-bit server SoC offering leading energy efficiency and performance and we expect that to continue into next year and beyond as we approach yet another generation of 14nm-based SoCs."

Though Intel is blustering that it has great capabilities here, we suspect that Google is doing the sums and figuring out if it could maybe split some large percentage of its workload off x86 and onto low-power highly customised ARM-driven packages. Though Intel offers some customization to large customers like Facebook and eBay (and probably Google), it does not yet afford them the flexibility granted by an ARM licence.

For Google, if it can take 20 percent of its several million compute nodes and switch them to a processor with drastically lower power consumption and/or bespoke capabilities, we reckon it may be worth it. Though all companies try to behave rationally, there's also some serious uber-boffin cred here that we reckon the company's employees would like.

Though Google has not confirmed the report, spokeswoman Liz Markman did tell El Reg: "We are actively engaged in designing the world's best infrastructure. This includes both hardware design (at all levels) and software design."

It seems that for sufficiently large tech companies, there may be some advantages stemming from designing their own chips; it's likely the technology will be sub-par compared to Intel or AMD, in terms of general-purpose raw performance, but the lower power bill – and the opportunity to customise the silicon to accelerate particular tasks – may be enough to motivate a move.

There is one other reason why these rumors are circulating – to keep market-leader Intel honest. "The reason we're rooting for the ARM ecosystem so much is that this is the healthiest the ecosystem has been since AMD made the Opteron," Facebook's hardware design chief Frank Frankovsky said recently. ®