Feeds

App Engine: Google's deepest secrets as a service

The software scales. But will the Google rulebook?

Top 5 reasons to deploy VMware with Tegile

An open source project in Google clothing

The Datastore API and other proprietary Google APIs present an additional problem. Because Google's infrastructure is closed source, you may be forced to rewrite your application if you ever move it off App Engine. "App Engine is one of the only platform-as-a-services out there that force you to use proprietary APIs," says Lucas Carlson, the founder of PHP Fog, a platform-as-a-service for PHP developers. "You do have to use their APIs very strictly."

Yes, databases like MongoDB and HBase are similar to BigTable, but there are still differences. If you move your application to one of these platforms, says Lynch, "you're not just going to take the WAR file and drop it in and have it run. There is going to be a little bit of moving things about. But the concepts shouldn't change, the basic way your app is laid out."

Dwight Merriman, who built MongoDB, goes a little further. "The APIs to the App Engine Datastore are different from the APIs to, say, HBase," he says. "The data models are similar, but you would have to recode. And in addition, you're probably using this whole API set that's App Engine-specific. So, if you write an App Engine app, it's going to be fairly difficult to port it off of App Engine. That's one of the problems of platform-as-a-service."

Lynch says he looks forward to the day when the non-relational world gets something like the JDBC (Java Database Connectivity) API , an API thats let you easily move between disparate databases. And such work is already underway, with projects like Django-norel.

But for App Engine users, the most important independent projects lie elsewhere. There are not one but two efforts to create an open source version of App Engine: App Scale and TyphoonAE. Since Google's underlying software is proprietary, these can never provide an exact replication, but each seeks to expose the same APIs so you can readily move entire applications from App Engine to other cloud services such as Amazon EC2 – or to your local servers. TyphoonAE replicates App Engine's Python side, while App Scale aims for Python and Java, and even Go.

The App Scale project was started at the University of California, Santa Barbara, by professors Chandra Krintz and Rich Wolski. Previously, Wolski had founded the Eucalyptus project, an effort to create an open source version of Amazon's cloud. "Eucalyptus has really taken off, building an open source community," Krintz tells us. "Taking that as inspiration, we decided to build an open source implementation of the APIs for Google App Engine as Eucalyptus did for Amazon's APIs. That way, we could have App Engine applications running on private clusters and experiment with new clusters and – this being a research platform – investigate with how cloud applications behave.

"We just implemented each API incrementally and, over time, developed a full-fledged distributed system such that you can now run your App Engine application either locally or on Google's resources and not know the difference. You don't have to change your application."

Krintz acknowledges that App Scale will never scale quite like App Engine. But she's confident it will eventually perform well enough for the enterprise. A new release is due later this summer. "With this next release, we're quite mature," says Krintz. "There are still bugs to be worked out, but it's pretty stable, and it's really scalable."

Chandra Krintz

Chandra Krintz

If the world is to run App Engine applications outside of App Engine, projects like this are the only option. Sean Lynch makes it clear that Google has no intention of offering its own "on-premise" version of App Engine, seeing limited value in platforms such as App Scale. "They may make deployment internally easier. You may be able to the same tool integration to get an app up and running," he says. "But there's still someone who's doing resource budgeting and making sure you have a capacity plan for the next year, worrying about versions and operating systems. If there's someone still worrying about that, you're not getting the benefit of a cloud-based platform-as-a-service."

When they started the App Scale project, after applying for a Google Research Award, Wolski and Krintz received seed money from the company. And on from the App Engine mailing list, when App Scale's engineers have questions about the public side of App Engine, Googlers are happy to help. After Google introduced the Go SDK for App Engine, an App Scale researcher asked for the SDK source code, and Google was happy to provide it. But the company had no interest in helping the open sourcers rebuild the Go incarnation of App Engine. And none too surprisingly, Google provides precious little information about the inner workings of App Engine.

"We had to infer everything," Krintz says. "They helped us in no way to understand any of the APIs. We had to emulate and run our own programs on their machines so that we could then implement the same functionality."

If it wasn't already obvious: Google believes everything should happen on the web.

Beginner's guide to SSL certificates

More from The Register

next story
It's Big, it's Blue... it's simply FABLESS! IBM's chip-free future
Or why the reversal of globalisation ain't gonna 'appen
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
Microsoft and Dell’s cloud in a box: Instant Azure for the data centre
A less painful way to run Microsoft’s private cloud
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
CAGE MATCH: Microsoft, Dell open co-located bit barns in Oz
Whole new species of XaaS spawning in the antipodes
AWS pulls desktop-as-a-service from the PC
Support for PCoIP protocol means zero clients can run cloudy desktops
prev story

Whitepapers

Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.