Feeds

App Engine: Google's deepest secrets as a service

The software scales. But will the Google rulebook?

3 Big data security analytics techniques

Google will never open source its back end. You'll never run the Google File System or Google MapReduce or Google BigTable on your own servers. Except on the rarest of occasions, the company won't even discuss the famously distributed software that underpins its sweeping collection of web services.

But if you like, you can still run your own applications atop GFS and MapReduce and BigTable. With Google App Engine – the "platform cloud" the company floated in the spring of 2008 – anyone can hoist code onto Google's live infrastructure. App Engine aims to share the company's distributed computing expertise by way of an online service.

"Google has built up this infrastructure – a lot of distributed software, internal processes – for building-out and scaling applications. We don't have the luxury of slowly ramping something up: when we launch something, we have to get it out there and scale it very, very quickly," Google App Engine product manager Sean Lynch recently told The Register.

"We decided we could take a lot of this infrastructure and expose it in a way that would let third-party developers use it – leverage the knowledge and techniques we have built up – to basically simplify the entire process of building their own web apps: building them, managing them once they're up there, and scaling them once they take off."

For some coders, it's an appealing proposition, not only because the Google back end has a reputation few others can match – a reputation fueled in part by the company's reluctance to discuss particulars – but also because App Engine completely removes the need to run your own infrastructure. As a platform cloud, App Engine goes several steps beyond an "infrastructure cloud" à la Amazon EC2. It doesn't give you raw virtual machines. It gives you APIs, and once you code to these APIs, the service takes care of the rest.

"When you're one guy trying to run a startup, you have to do absolutely everything. I've done everything from managing machines to writing code to marketing," says Jeff Schnitzer, a former senior engineer in EA's online games division who's now using App Engine to build an online-dating application. "That's one of the reasons why I've been drawn to App Engine. It eliminates entire classes of job descriptions, from sys admin to DBA."

The rub is that in order to benefit from this automation, you have to play by a strict Google rulebook. All applications must be built with Python, Java, or one of a handful of other supported languages, including Google's new-age Go programming language, which was just added to the list. And even within these languages, there are limits on the libraries and frameworks you can use, the way you handle data, and the duration of your processes.

Google is loosening some restrictions as the service matures. But inherently, App Engine requires a change of mindset. "You have to throw away a lot of what you know and basically write for Google's model: small instances, faster start, completely new data storage. It's a completely different beast," says Matt Cooper, a developer and entrepreneur who just recently started using the service. And because Google jealously guards the secrets of its infrastructure, anyone who builds an application atop App Engine will face additional hurdles if they ever decide to move the app elsewhere.

All of which makes Google App Engine a particularly fascinating case study. Much like in other markets, Google is promising you an added payoff if you seriously change the way you've done things in the past – and if you put a hefty amount of trust on its servers. Many have already embraced the proposition – App Engine serves more than a billion and a half page views a day, and 100,000 coders access the online console each month – but it's yet to be seen whether the service has a future in the mainstream.

App Engine Logo

Google App Engine

We know that Google has a knack for scaling web applications. "Google employees are very good about not talking about their proprietary [infrastructure], but they do give you qualitative feel for it," says Dwight Merriman, cofounder of MongoDB, the distributed database that seeks to solve many of the same scaling issues as the Google back end. What we don't know is whether the Google ethos can scale into the enterprise.

Google believes it can. Lynch calls App Engine a "long-term business", and later this year, the service will exit a three-year beta period, brandishing new enterprise-centric terms of service. Proof of the service's viability, the company believes, lies in the track record of the Google infrastructure. "This is a service that's a tremendous differentiator for us, " says Lynch. "We can see it being valuable for other people to use it as well."

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
BOFH: Oh DO tell us what you think. *CLICK*
$%%&amp Oh dear, we've been cut *CLICK* Well hello *CLICK* You're breaking up...
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.