Feeds

NASA and Rackspace open source cloud fluffer

OpenStack targets one million machine Nebula

Internet Security Threat Report 2014

NASA and Rackspace have joined forces to open source a new platform for building so-called infrastructure clouds. Known as OpenStack, the platform is available under an Apache license, and when completed – possibly by the end of the year – NASA and Rackspace will ditch their current infrastructure cloud platforms, which don't scale as they would like.

NASA's Nebula infrastructure cloud is installed installed at NASA's Ames Research Center and is in the process of being and extended to the Goddard Spaceflight Center outside of Washington, DC. The Nebula cloud was spearheaded by Chris Kemp, who is now NASA's chief technology officer, and in an interview with El Reg, Kemp had nice things to say about the Eucalyptus cloud fabric controller that Nebula was currently using, which is an open source clone of Amazon's EC2 compute cloud and S3 storage cloud.

But Kemp also said that the scalability of the product and other issues with Eucalyptus (including the inability by NASA to get some of its enhancements into the Eucalyptus code base) compelled Kemp to take the entire Nebula team and dedicate it – for the past six months – to creating a new fabric controller, called Nova, from scratch.

It was serendipity that at just the time when NASA was working on the guts of a new compute engine that Rackspace had decided to contact NASA about its plan to open source the "Ozone" code behind its Cloud Servers compute cloud and the Cloud Files storage cloud (formerly known as CloudFS). NASA had released the Nova cloud fabric controller under an Apache 2 license at http://novacc.org/ (the code is currently at the 0.42 release level and admittedly buggy), and according to Jim Curry, vice president of corporate development at Rackspace, decided not only to open source its Cloud Servers code but to start the OpenStack project to mix-and-match the best bits of its Ozone code base with NASA's Nova code base to forge a better cloud fabric controller.

Nova is written in C, C++, and Python and Ozone is written in Python, and at this point, the exact nature of the merged project is being hammered out by NASA, Rackspace, and 25 other partners who are lining up to support the OpenStack Compute fabric controller.

Like the Eucalyptus tools NASA is currently using in its Nebula cloud, the Nova cloud fabric controller is built to mimic Amazon's EC2 and S3 APIs. Most of the code is written in Python, using the Tornado and Twisted frameworks. It uses the open source Advanced Message Queuing Protocol (AMQP) that Red Hat has championed for its Enterprise MRG cloudy Linux stack as well as a Redis distributed key value store. The code was released at the end of May.

NASA will be adopting the OpenStack Object Storage as its default object-based storage, and this code is literally Rackspace's Cloud Files opened up under an Apache 2 license to match that of NASA's Nova fabric controller.

Sometime in the fourth quarter, after the OpenStack project gets fully organized and programmers from inside NASA and Rackspace as well as from the OpenStack community decide what features and standards to support, both NASA and Rackspace intend to put the OpenStack Compute and Object Storage code into production. At that point, Eucalyptus and VMware's vSphere will be relegated to jobs where NASA needs to support a cloud that has third party application certification, something that Kemp never envisioned doing on the Nebula cloud in the first place and still doesn't see happening. "There is a place in the enterprise for all of these platforms," says Kemp.

The problem with the tools being created and sold by Eucalyptus Systems, VMware, and others trying to provide tools to fluff up clouds is that the scale that NASA and Rackspace are wrestling with is beyond the capability of these products.

"We are not the target customer," says Kemp. "We are constantly working around issues and creating code because of this." The difference is between a platform, which you can pull apart in a modular way – perhaps changing the networking engine or the queuing engine to boost performance – and a product, which is tends to be a single, integrated thing bound by the needs of an average of customers, not those pushing the scalability limits.

What kind of scalability are NASA and Rackspace talking about with the OpenStack software? The initial design goals agreed to by the two parties (who do not have any kind of formal arrangement except to contribute code to the OpenStack project and work in the community) is for OpenStack to span 1 million host machines and control up to 60 million virtual machines.

"VMware has not built something that can scale up to Amazon-level scalability of hundreds of thousands to millions of servers," says Curry at Rackspace. And as for Eucalyptus, Kemp over at NASA said "our team's analysis is that it could not get anywhere near it," meaning the design goals for OpenStack. Curry joked that if anyone wants to provide Rackspace with the iron, it would be happy to test it across a million boxes. (What do you say, Google?)

The exact feeds and speeds of the OpenStack Compute fabric controller have not been hammered out, but Rackspace hosted a design summit in Austin, Texas from July 13 through 16 where 100 technical advisors from 25 companies were on hard to help hammer out the initial details. These include AMD, Autonomic Resources, Citrix Systems, Cloud.com, Cloudkick, CloudSwitch, Dell, enStratus, FathomDB, Intel, iomart Group, Limelight, Nicira, NTT Data, Opscode, Peer 1, Puppet Labs, RightScale, Riptano, Scalr, Sonian, Spiceworks, ThoughtWorks, Zenoss, and Zuora. The initial cloud fabric will support KVM virtual machines as well as Xen virtual machines based on the open source Xen as well as Citrix Systems' XenServer implementation. (It is not clear yet how the Xen community, which created its own cloud stack last summer, and Citrix, which sells a commercial cloud fluffer called Citrix Cloud Center (or C3 for short), will participate in the CloudStack project.)

Internet Security Threat Report 2014

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
IBM storage revenues sink: 'We are disappointed,' says CEO
Time to put the storage biz up for sale?
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
Symantec backs out of Backup Exec: Plans to can appliance in Jan
Will still provide support to existing customers
VMware's tool to harden virtual networks: a spreadsheet
NSX security guide lands in intriguing format
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.