Feeds

App Engine: Google's deepest secrets as a service

The software scales. But will the Google rulebook?

Next gen security for virtualised datacentres

Google will never open source its back end. You'll never run the Google File System or Google MapReduce or Google BigTable on your own servers. Except on the rarest of occasions, the company won't even discuss the famously distributed software that underpins its sweeping collection of web services.

But if you like, you can still run your own applications atop GFS and MapReduce and BigTable. With Google App Engine – the "platform cloud" the company floated in the spring of 2008 – anyone can hoist code onto Google's live infrastructure. App Engine aims to share the company's distributed computing expertise by way of an online service.

"Google has built up this infrastructure – a lot of distributed software, internal processes – for building-out and scaling applications. We don't have the luxury of slowly ramping something up: when we launch something, we have to get it out there and scale it very, very quickly," Google App Engine product manager Sean Lynch recently told The Register.

"We decided we could take a lot of this infrastructure and expose it in a way that would let third-party developers use it – leverage the knowledge and techniques we have built up – to basically simplify the entire process of building their own web apps: building them, managing them once they're up there, and scaling them once they take off."

For some coders, it's an appealing proposition, not only because the Google back end has a reputation few others can match – a reputation fueled in part by the company's reluctance to discuss particulars – but also because App Engine completely removes the need to run your own infrastructure. As a platform cloud, App Engine goes several steps beyond an "infrastructure cloud" à la Amazon EC2. It doesn't give you raw virtual machines. It gives you APIs, and once you code to these APIs, the service takes care of the rest.

"When you're one guy trying to run a startup, you have to do absolutely everything. I've done everything from managing machines to writing code to marketing," says Jeff Schnitzer, a former senior engineer in EA's online games division who's now using App Engine to build an online-dating application. "That's one of the reasons why I've been drawn to App Engine. It eliminates entire classes of job descriptions, from sys admin to DBA."

The rub is that in order to benefit from this automation, you have to play by a strict Google rulebook. All applications must be built with Python, Java, or one of a handful of other supported languages, including Google's new-age Go programming language, which was just added to the list. And even within these languages, there are limits on the libraries and frameworks you can use, the way you handle data, and the duration of your processes.

Google is loosening some restrictions as the service matures. But inherently, App Engine requires a change of mindset. "You have to throw away a lot of what you know and basically write for Google's model: small instances, faster start, completely new data storage. It's a completely different beast," says Matt Cooper, a developer and entrepreneur who just recently started using the service. And because Google jealously guards the secrets of its infrastructure, anyone who builds an application atop App Engine will face additional hurdles if they ever decide to move the app elsewhere.

All of which makes Google App Engine a particularly fascinating case study. Much like in other markets, Google is promising you an added payoff if you seriously change the way you've done things in the past – and if you put a hefty amount of trust on its servers. Many have already embraced the proposition – App Engine serves more than a billion and a half page views a day, and 100,000 coders access the online console each month – but it's yet to be seen whether the service has a future in the mainstream.

App Engine Logo

Google App Engine

We know that Google has a knack for scaling web applications. "Google employees are very good about not talking about their proprietary [infrastructure], but they do give you qualitative feel for it," says Dwight Merriman, cofounder of MongoDB, the distributed database that seeks to solve many of the same scaling issues as the Google back end. What we don't know is whether the Google ethos can scale into the enterprise.

Google believes it can. Lynch calls App Engine a "long-term business", and later this year, the service will exit a three-year beta period, brandishing new enterprise-centric terms of service. Proof of the service's viability, the company believes, lies in the track record of the Google infrastructure. "This is a service that's a tremendous differentiator for us, " says Lynch. "We can see it being valuable for other people to use it as well."

The essential guide to IT transformation

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
Oracle reveals 32-core, 10 BEEELLION-transistor SPARC M7
New chip scales to 1024 cores, 8192 threads 64 TB RAM, at speeds over 3.6GHz
US regulators OK sale of IBM's x86 server biz to Lenovo
Now all that remains is for gov't offices to ban the boxes
Object storage bods Exablox: RAID is dead, baby. RAID is dead
Bring your own disks to its object appliances
Nimble's latest mutants GORGE themselves on unlucky forerunners
Crossing Sandy Bridges without stopping for breath
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
7 Elements of Radically Simple OS Migration
Avoid the typical headaches of OS migration during your next project by learning about 7 elements of radically simple OS migration.
BYOD's dark side: Data protection
An endpoint data protection solution that adds value to the user and the organization so it can protect itself from data loss as well as leverage corporate data.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?