NASA and Rackspace open source cloud fluffer

OpenStack targets one million machine Nebula

Designing a Defense for Mobile Applications

NASA and Rackspace have joined forces to open source a new platform for building so-called infrastructure clouds. Known as OpenStack, the platform is available under an Apache license, and when completed – possibly by the end of the year – NASA and Rackspace will ditch their current infrastructure cloud platforms, which don't scale as they would like.

NASA's Nebula infrastructure cloud is installed installed at NASA's Ames Research Center and is in the process of being and extended to the Goddard Spaceflight Center outside of Washington, DC. The Nebula cloud was spearheaded by Chris Kemp, who is now NASA's chief technology officer, and in an interview with El Reg, Kemp had nice things to say about the Eucalyptus cloud fabric controller that Nebula was currently using, which is an open source clone of Amazon's EC2 compute cloud and S3 storage cloud.

But Kemp also said that the scalability of the product and other issues with Eucalyptus (including the inability by NASA to get some of its enhancements into the Eucalyptus code base) compelled Kemp to take the entire Nebula team and dedicate it – for the past six months – to creating a new fabric controller, called Nova, from scratch.

It was serendipity that at just the time when NASA was working on the guts of a new compute engine that Rackspace had decided to contact NASA about its plan to open source the "Ozone" code behind its Cloud Servers compute cloud and the Cloud Files storage cloud (formerly known as CloudFS). NASA had released the Nova cloud fabric controller under an Apache 2 license at http://novacc.org/ (the code is currently at the 0.42 release level and admittedly buggy), and according to Jim Curry, vice president of corporate development at Rackspace, decided not only to open source its Cloud Servers code but to start the OpenStack project to mix-and-match the best bits of its Ozone code base with NASA's Nova code base to forge a better cloud fabric controller.

Nova is written in C, C++, and Python and Ozone is written in Python, and at this point, the exact nature of the merged project is being hammered out by NASA, Rackspace, and 25 other partners who are lining up to support the OpenStack Compute fabric controller.

Like the Eucalyptus tools NASA is currently using in its Nebula cloud, the Nova cloud fabric controller is built to mimic Amazon's EC2 and S3 APIs. Most of the code is written in Python, using the Tornado and Twisted frameworks. It uses the open source Advanced Message Queuing Protocol (AMQP) that Red Hat has championed for its Enterprise MRG cloudy Linux stack as well as a Redis distributed key value store. The code was released at the end of May.

NASA will be adopting the OpenStack Object Storage as its default object-based storage, and this code is literally Rackspace's Cloud Files opened up under an Apache 2 license to match that of NASA's Nova fabric controller.

Sometime in the fourth quarter, after the OpenStack project gets fully organized and programmers from inside NASA and Rackspace as well as from the OpenStack community decide what features and standards to support, both NASA and Rackspace intend to put the OpenStack Compute and Object Storage code into production. At that point, Eucalyptus and VMware's vSphere will be relegated to jobs where NASA needs to support a cloud that has third party application certification, something that Kemp never envisioned doing on the Nebula cloud in the first place and still doesn't see happening. "There is a place in the enterprise for all of these platforms," says Kemp.

The problem with the tools being created and sold by Eucalyptus Systems, VMware, and others trying to provide tools to fluff up clouds is that the scale that NASA and Rackspace are wrestling with is beyond the capability of these products.

"We are not the target customer," says Kemp. "We are constantly working around issues and creating code because of this." The difference is between a platform, which you can pull apart in a modular way – perhaps changing the networking engine or the queuing engine to boost performance – and a product, which is tends to be a single, integrated thing bound by the needs of an average of customers, not those pushing the scalability limits.

What kind of scalability are NASA and Rackspace talking about with the OpenStack software? The initial design goals agreed to by the two parties (who do not have any kind of formal arrangement except to contribute code to the OpenStack project and work in the community) is for OpenStack to span 1 million host machines and control up to 60 million virtual machines.

"VMware has not built something that can scale up to Amazon-level scalability of hundreds of thousands to millions of servers," says Curry at Rackspace. And as for Eucalyptus, Kemp over at NASA said "our team's analysis is that it could not get anywhere near it," meaning the design goals for OpenStack. Curry joked that if anyone wants to provide Rackspace with the iron, it would be happy to test it across a million boxes. (What do you say, Google?)

The exact feeds and speeds of the OpenStack Compute fabric controller have not been hammered out, but Rackspace hosted a design summit in Austin, Texas from July 13 through 16 where 100 technical advisors from 25 companies were on hard to help hammer out the initial details. These include AMD, Autonomic Resources, Citrix Systems, Cloud.com, Cloudkick, CloudSwitch, Dell, enStratus, FathomDB, Intel, iomart Group, Limelight, Nicira, NTT Data, Opscode, Peer 1, Puppet Labs, RightScale, Riptano, Scalr, Sonian, Spiceworks, ThoughtWorks, Zenoss, and Zuora. The initial cloud fabric will support KVM virtual machines as well as Xen virtual machines based on the open source Xen as well as Citrix Systems' XenServer implementation. (It is not clear yet how the Xen community, which created its own cloud stack last summer, and Citrix, which sells a commercial cloud fluffer called Citrix Cloud Center (or C3 for short), will participate in the CloudStack project.)

The Power of One eBook: Top reasons to choose HP BladeSystem

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
Attack of the clones: Oracle's latest Red Hat Linux lookalike arrives
Oracle's Linux boss says Larry's Linux isn't just for Oracle apps anymore
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
prev story


Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.