Feeds

App Engine: Google's deepest secrets as a service

The software scales. But will the Google rulebook?

3 Big data security analytics techniques

An open source project in Google clothing

The Datastore API and other proprietary Google APIs present an additional problem. Because Google's infrastructure is closed source, you may be forced to rewrite your application if you ever move it off App Engine. "App Engine is one of the only platform-as-a-services out there that force you to use proprietary APIs," says Lucas Carlson, the founder of PHP Fog, a platform-as-a-service for PHP developers. "You do have to use their APIs very strictly."

Yes, databases like MongoDB and HBase are similar to BigTable, but there are still differences. If you move your application to one of these platforms, says Lynch, "you're not just going to take the WAR file and drop it in and have it run. There is going to be a little bit of moving things about. But the concepts shouldn't change, the basic way your app is laid out."

Dwight Merriman, who built MongoDB, goes a little further. "The APIs to the App Engine Datastore are different from the APIs to, say, HBase," he says. "The data models are similar, but you would have to recode. And in addition, you're probably using this whole API set that's App Engine-specific. So, if you write an App Engine app, it's going to be fairly difficult to port it off of App Engine. That's one of the problems of platform-as-a-service."

Lynch says he looks forward to the day when the non-relational world gets something like the JDBC (Java Database Connectivity) API , an API thats let you easily move between disparate databases. And such work is already underway, with projects like Django-norel.

But for App Engine users, the most important independent projects lie elsewhere. There are not one but two efforts to create an open source version of App Engine: App Scale and TyphoonAE. Since Google's underlying software is proprietary, these can never provide an exact replication, but each seeks to expose the same APIs so you can readily move entire applications from App Engine to other cloud services such as Amazon EC2 – or to your local servers. TyphoonAE replicates App Engine's Python side, while App Scale aims for Python and Java, and even Go.

The App Scale project was started at the University of California, Santa Barbara, by professors Chandra Krintz and Rich Wolski. Previously, Wolski had founded the Eucalyptus project, an effort to create an open source version of Amazon's cloud. "Eucalyptus has really taken off, building an open source community," Krintz tells us. "Taking that as inspiration, we decided to build an open source implementation of the APIs for Google App Engine as Eucalyptus did for Amazon's APIs. That way, we could have App Engine applications running on private clusters and experiment with new clusters and – this being a research platform – investigate with how cloud applications behave.

"We just implemented each API incrementally and, over time, developed a full-fledged distributed system such that you can now run your App Engine application either locally or on Google's resources and not know the difference. You don't have to change your application."

Krintz acknowledges that App Scale will never scale quite like App Engine. But she's confident it will eventually perform well enough for the enterprise. A new release is due later this summer. "With this next release, we're quite mature," says Krintz. "There are still bugs to be worked out, but it's pretty stable, and it's really scalable."

Chandra Krintz

Chandra Krintz

If the world is to run App Engine applications outside of App Engine, projects like this are the only option. Sean Lynch makes it clear that Google has no intention of offering its own "on-premise" version of App Engine, seeing limited value in platforms such as App Scale. "They may make deployment internally easier. You may be able to the same tool integration to get an app up and running," he says. "But there's still someone who's doing resource budgeting and making sure you have a capacity plan for the next year, worrying about versions and operating systems. If there's someone still worrying about that, you're not getting the benefit of a cloud-based platform-as-a-service."

When they started the App Scale project, after applying for a Google Research Award, Wolski and Krintz received seed money from the company. And on from the App Engine mailing list, when App Scale's engineers have questions about the public side of App Engine, Googlers are happy to help. After Google introduced the Go SDK for App Engine, an App Scale researcher asked for the SDK source code, and Google was happy to provide it. But the company had no interest in helping the open sourcers rebuild the Go incarnation of App Engine. And none too surprisingly, Google provides precious little information about the inner workings of App Engine.

"We had to infer everything," Krintz says. "They helped us in no way to understand any of the APIs. We had to emulate and run our own programs on their machines so that we could then implement the same functionality."

If it wasn't already obvious: Google believes everything should happen on the web.

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
BOFH: Oh DO tell us what you think. *CLICK*
$%%&amp Oh dear, we've been cut *CLICK* Well hello *CLICK* You're breaking up...
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.