Feeds

App Engine: Google's deepest secrets as a service

The software scales. But will the Google rulebook?

7 Elements of Radically Simple OS Migration

Google will never open source its back end. You'll never run the Google File System or Google MapReduce or Google BigTable on your own servers. Except on the rarest of occasions, the company won't even discuss the famously distributed software that underpins its sweeping collection of web services.

But if you like, you can still run your own applications atop GFS and MapReduce and BigTable. With Google App Engine – the "platform cloud" the company floated in the spring of 2008 – anyone can hoist code onto Google's live infrastructure. App Engine aims to share the company's distributed computing expertise by way of an online service.

"Google has built up this infrastructure – a lot of distributed software, internal processes – for building-out and scaling applications. We don't have the luxury of slowly ramping something up: when we launch something, we have to get it out there and scale it very, very quickly," Google App Engine product manager Sean Lynch recently told The Register.

"We decided we could take a lot of this infrastructure and expose it in a way that would let third-party developers use it – leverage the knowledge and techniques we have built up – to basically simplify the entire process of building their own web apps: building them, managing them once they're up there, and scaling them once they take off."

For some coders, it's an appealing proposition, not only because the Google back end has a reputation few others can match – a reputation fueled in part by the company's reluctance to discuss particulars – but also because App Engine completely removes the need to run your own infrastructure. As a platform cloud, App Engine goes several steps beyond an "infrastructure cloud" à la Amazon EC2. It doesn't give you raw virtual machines. It gives you APIs, and once you code to these APIs, the service takes care of the rest.

"When you're one guy trying to run a startup, you have to do absolutely everything. I've done everything from managing machines to writing code to marketing," says Jeff Schnitzer, a former senior engineer in EA's online games division who's now using App Engine to build an online-dating application. "That's one of the reasons why I've been drawn to App Engine. It eliminates entire classes of job descriptions, from sys admin to DBA."

The rub is that in order to benefit from this automation, you have to play by a strict Google rulebook. All applications must be built with Python, Java, or one of a handful of other supported languages, including Google's new-age Go programming language, which was just added to the list. And even within these languages, there are limits on the libraries and frameworks you can use, the way you handle data, and the duration of your processes.

Google is loosening some restrictions as the service matures. But inherently, App Engine requires a change of mindset. "You have to throw away a lot of what you know and basically write for Google's model: small instances, faster start, completely new data storage. It's a completely different beast," says Matt Cooper, a developer and entrepreneur who just recently started using the service. And because Google jealously guards the secrets of its infrastructure, anyone who builds an application atop App Engine will face additional hurdles if they ever decide to move the app elsewhere.

All of which makes Google App Engine a particularly fascinating case study. Much like in other markets, Google is promising you an added payoff if you seriously change the way you've done things in the past – and if you put a hefty amount of trust on its servers. Many have already embraced the proposition – App Engine serves more than a billion and a half page views a day, and 100,000 coders access the online console each month – but it's yet to be seen whether the service has a future in the mainstream.

App Engine Logo

Google App Engine

We know that Google has a knack for scaling web applications. "Google employees are very good about not talking about their proprietary [infrastructure], but they do give you qualitative feel for it," says Dwight Merriman, cofounder of MongoDB, the distributed database that seeks to solve many of the same scaling issues as the Google back end. What we don't know is whether the Google ethos can scale into the enterprise.

Google believes it can. Lynch calls App Engine a "long-term business", and later this year, the service will exit a three-year beta period, brandishing new enterprise-centric terms of service. Proof of the service's viability, the company believes, lies in the track record of the Google infrastructure. "This is a service that's a tremendous differentiator for us, " says Lynch. "We can see it being valuable for other people to use it as well."

Best practices for enterprise data

More from The Register

next story
Microsoft's Euro cloud darkens: US FEDS can dig into foreign servers
They're not emails, they're business records, says court
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
VMware builds product executables on 50 Mac Minis
And goes to the Genius Bar for support
Multipath TCP speeds up the internet so much that security breaks
Black Hat research says proposed protocol will bork network probes, flummox firewalls
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Microsoft says 'weird things' can happen during Windows Server 2003 migrations
Fix coming for bug that makes Kerberos croak when you run two domain controllers
Cisco says network virtualisation won't pay off everywhere
Another sign of strain in the Borg/VMware relationship?
prev story

Whitepapers

7 Elements of Radically Simple OS Migration
Avoid the typical headaches of OS migration during your next project by learning about 7 elements of radically simple OS migration.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
A new approach to endpoint data protection
What is the best way to ensure comprehensive visibility, management, and control of information on both company-owned and employee-owned devices?