Google demanding Intel's hottest chips?

Original URL: https://www.theregister.com/2008/10/15/google_and_intel/

Inside Project Will Power

Posted in Channel, 15th October 2008 00:27 GMT

When purchasing server processors directly from Intel, Google has insisted on a guarantee that the chips can operate at temperatures five degrees centigrade higher than their standard qualification, according to a former Google employee. This allowed the search giant to maintain higher temperatures within its data centers, the ex-employee says, and save millions of dollars each year in cooling costs.

If the chips failed prematurely at these higher temperatures, the former Googler says, Intel was obliged to replace them at no extra charge.

Intel denies this was ever the case. "This is NOT true," a company spokesman said in an email. Google declined to comment on its relationship with Intel. "Google invests heavily in technical facilities and has dozens of facilities around the world with many computers," reads a statement from the company. "However, we don't disclose details about our infrastructure or supplier relationships."

The ex-Google employee learned of this Intel pact a little more than a year ago, during a Google "Tech Talk" open to anyone at the company. The talk was given by a Google thermal dynamics engineer, part of a small team - perhaps no larger than two people - that oversees heat issues inside the company's data centers.

According to the same ex-employee, it is now the norm for Google to construct its data centers by piecing together intermodal shipping containers pre-packed with servers and cooling equipment. In 2003, Google filed for a patent on this sort of modular data center, and the patent was granted last October.

This modular setup - known internally as Project Will Power - allows Google to construct the building blocks for each data center at a central location and then ship them around the world as needed. Robert X Cringely first leaked word of Google's modular data center work in 2005, claiming the project began after representatives of the Internet Archive pitched the idea to Google co-founder Larry Page.

Heat Saves

It's no secret that Google builds its own data centers - and many of the servers within. It's long been said that after the top server vendors, the ad broker consumes more processors and hard disks than anyone else in the world - and Intel freely acknowledges that it provides at least some of the chips.

Intel also customizes power saving motherboards for Google - a service not afforded other customers. But our source tells us that any motherboard pact is separate from the companies' high temperature chip agreement.

If Google can leverage an extra five degrees centigrade from Intel, it can save a few penguins - and some serious cash. According to Data Center Knowledge, Google runs at least 36 data centers across the globe, and several more are under construction. One of the newer sites, in The Dalles, Oregon, includes three data center buildings, each measuring 68,680 square feet.

According to Mark Monroe, Sun Microsystems' director of sustainable computing, data center managers can save 4 per cent in energy costs for every degree (Fahrenheit) they raise the thermostat. If you run your data center at a higher temp, you use less power and spend less money on the equipment needed to cool it down.

But if you raise the thermostat, you may void your hardware warranties. "The hardware manufacturer will usually have a temperature range," says Rich Miller, editor of Data Center Knowledge. "If the equipment is running outside of the range, the manufacturer might be inclined to challenge any sort of returns or credits. That's the thinking across the industry."

Google cranks the thermostat

In a story published today at Data Center Knowledge, Google recommends operating data centers at higher temperatures than the norm. "The guidance we give to data center operators is to raise the thermostat," Google energy program manager Erik Teetzel told Data Center Knowledge. "Many data centers operate at 70 [Fahrenheit] degrees or below. We’d recommend looking at going to 80 [Fahrenheit] degrees."

On August 1, the American Society of Heating, Refrigerating, and Air-Conditioning Engineers (ASHRAE) raised its recommended data center temperature range to between 68 and 77 degrees Fahrenheit (20 to 25 degrees Celsius). This recommendation is backed by 17 industry players, including IBM, Cisco, and Intel. "Most data centers tend to operate between 68 and 70 degrees [Fahrenheit]," says Fred Stack, an Emerson Network Power who heads ASRAE's data center committee. "But ASHRAE is actively promoting the increase of that number, along with server OEMs."

According to Stack, it's well known that Google operates its data centers at high temperatures. "Google has gone beyond the 77 degree point where most data centers start speeding up their server fans," he says. "There's no question Google operates its servers in warmer environments than the general [data center] population. Google doesn't talk about this but there are enough rumors in the world that I'm quite sure of this..."

"You can get computers to operate in environments that are well above 77 degrees. The military does it all the time."

Sorting It Out

Stack would not be surprised if Google also had a pact with Intel that upped the temperature qualifications of its processors. Processor temperature qualifications deal with the temperature of the chip itself, as opposed to the ambient temperature of the data center. So, whereas the data center might be cooled to 25 degrees Celsius, the chip itself might run 55 degrees.

"[Google uses] so many servers, they can command something special for their specific application," Stack says. "I would see no reason that Google couldn't get an Intel or AMD to commit to a special selection of components to meet a higher requirement. The military does it all the time."

In other words, he's speculating that Intel would use a special sort routine that would select chips more qualified than others to deal with high temperatures. "You could come up with a test routine that would test the ability of processors to withstand heat," he says.

AMD tells us that currently, it does not provide such a service to any of its customers. "Right now, our business just isn't set up to do that sort of thing," says Brent Kerby, product marketing manager for AMD's Opteron chips. But the company says that it would be able to do special sorting if a particular "business case" warranted it.

In selling chips to Google, Intel is fending off competition not just from AMD, but also the big-name server manufacturers. As recently as this week, Dell lamented its inability to land Google as a server customer. And privately, the big server OEMs have complained that Intel distributes chips directly to Mountain View. If Intel is providing Google with specialized chip qualifications, the server brigade must wonder why Mountain View gets perks they don't. ®

Update: This story has been updated to amend AMD statements on special processor sorts.