Google and the myth of the open cloud
The world's most closed open company
It is a truth universally acknowledged that if Google offers the world a web service, large numbers of people will convince themselves that it's superior to anything else they can get their hands on - and less likely to condemn them to some sort of Redmondian future in which a single corporation has them in a metaphorical vice grip.
So it is with Google App Engine, the 18-month-old service that lets outside developers build and run web apps on the company's very own distributed infrastructure. According to the market research types at Evans Data, developers everywhere are convinced that App Engine will overtake Amazon in the race into the so-called public cloud. They've even decided that the Google cloud is their best bet for avoiding the dreaded "vendor lock-in."
You know the mindset. Google says it's open, so it must be open. Google says that it has opened up more than one million lines of code, that it's hosting more than 150,000 open source projects, that its web browser is open, and that its mobile OS is open - so its cloud must be open too. Its cloud must be much more accommodating than Amazon's.
Except that it's not.
Google's famously distributed infrastructure is in no way open. Whereas the likes of Facebook and Yahoo! run much of their back-end setup on open source code, Google builds its own proprietary platforms - from the Google File System (GFS) to the number-crunching MapReduce platform to the BigTable distributed database - and these are jealously guarded. Google is reluctant to even talk about them.
What's more, these Googly platforms place extensive restrictions on what you can build atop them. This is true whether you're an internal Google developer or an outsider tapping the App Engine. There's good reason for this. The idea is to fit all applications to predefined templates that can run across the company's worldwide network of data centers, so that performance can scale up (near-)instantly as needed.
"The question is: how do you actually get the applications to use the infrastructure? How do you distribute it? How do you optimize it? That's the hard part. To do that you require an insane amount of force of will," Google senior manager of engineering and architecture Vijay Gill told a cloudy conference this summer.
"People are lazy. They say 'I don't want to design my applications into this confined space.' And it is a confined space. So you need a large force of will to get people to do that."