Happy third birthday, OpenStack: Ready to dominate all clouds now?
'A testament to the power of bourbon and unit test coverage'
It has been three years since the techies at NASA and Rackspace Hosting mashed up their respective cloudy compute and storage software to create OpenStack.
And by most objective measures, the resulting cloud control freak has gained enough momentum - both in terms of partner and developer enthusiasm and the increasing sophistication of its code - to bypass Eucalyptus and CloudStack. It has been fighting against the two older open source alternatives for years, but now it looks like it is close to claiming the much-desired title of "the Linux of cloud operating systems".
OpenStack has come far very fast, and naturally elicits comparisons to Linux itself, which took the enterprise computing market by storm in the late 1990s, first in technical and high performance computing and then among enterprises at large, as a cost-effective alternative to Unix and proprietary operating systems.
Linux now drives about a quarter of server revenues each year and has eclipsed Unix as the main alternative to Microsoft's Windows Server platform. The familiarity that most businesses have with Windows on the desktop has been the key driver of Windows in the data center, much as the similarity between Linux and Unix has been a key driver of Linux systems sales.
In the cloud controller space, there are no such incumbents, although there are natural affinities between operating systems, hypervisors, and now cloud controllers. So few true clouds have actually been built to date (although some of them are certainly large, such as those puffed up by Amazon Web Services, Microsoft, Google, and Rackspace) that the market is still, to a certain extent, wide open.
The market for private clouds is particularly wide open, while that for public clouds is dominated by Linux and either Xen or KVM hypervisors with the exception of Microsoft Azure, which is based on an amalgam of the Hyper-V hypervisor, Windows Server, and System Center with some other goodies thrown in.
Some companies want one throat to choke, while others want the most openness and flexibility they can get. If you can have both, it would be ideal. But that's not how the IT industry works, now is it?
The OpenStack stack community still has a tremendous amount of work to do, and competing cloud platforms (at both the infrastructure layer and higher up in the platform layer in some cases) are not sitting still in terms of development and driving customer adoption.
But as OpenStack turns three, the community can certainly be proud of the fact that NASA and Rackspace managed to get this thing off the ground and, despite much grousing over Rackspace having too tight a hold on the project for the first two years, managing to place it in the hands of an independent foundation that is steering its development. Well, as much as anyone can be said to be steering a collective of open source projects.
The number of cumulative OpenStack developers continues to grow
"It has been three years that we could never have predicted, with just incredible growth and involvement from way more people than we ever expected," says Jonathan Bryce, executive director at the OpenStack Foundation, which controls the governance for the project.
Bryce told El Reg: "There have been some interesting criticisms along the way, but to me the thing that just really blow my mind is the people who have stepped up and volunteered to be part of the OpenStack community."
The official birthday celebration for OpenStack will take place in Portland on July 24, but there will be over 40 community birthday parties hosted around the world to commemorate the passing into year four for the cloud control freak.
Any excuse to drink, really. That may not be much of an exaggeration, either.
"The fact that we get along at all is a testament to the power of bourbon," explains Josh McKenty, CTO and co-founder of Piston Cloud. Piston Cloud is one of the several distributors of commercial-grade OpenStack support and McKenty was one of the chief techies at NASA who worked on the original Eucalyptus deployments for the space agency. He also worked on the Nova controller that NASA contributed to the OpenStack cause.
"The culture of OpenStack was dominated by drinking and unit test coverage," says McKenty. "The OpenStack Summit is basically Thanksgiving dinner. We have a couple of crazy uncles, people making inappropriate comments, others taking their clothes off. It really is Thanksgiving dinner."
That sure sounds a lot less boring than working for Microsoft or VMware. And, as Bryce explains, it has been edgy from Day One when Rackspace first started pondering what it might do in conjunction with NASA to try to create a pure open source cloud controller. OpenStack is now a genuine contender against Eucalyptus, which has some core components as closed source; Cloud.com, which has since been opened up by Citrix Systems under an Apache license; and VMware's vCloud cloud. The VMWare offering is an extension to its vSphere server virtualization suite - it adds orchestration, metering, and other features to turn it into a cloud.
Amazon ... schmamazon
The fact that OpenStack, unlike Eucalyptus and CloudStack, is not concerned with API compatibility with Amazon Web Services, was a big deal, and one that could have been a problem - and still might be in the future, depending on how dominant AWS becomes in cloud computing. (More on this in a minute.)
"I was at Rackspace when we launched this, and this was something that we had to go to the board of directors to get approval for because we were going to be giving away a lot of intellectual property as a public company," says Bryce.
"And the question, after they had gotten comfortable a bit with the idea, which took some time, that they asked us was this: 'What is the biggest risk? What could go wrong?' And I said that the worst-case scenario is that we go through all the trouble of making our code public and trying to do development in the open and nobody else shows up."
Bring a bottle - and your code contribution
Well, people showed up to the OpenStack party. As of this week, the OpenStack community has 10,149 members from 120 different countries. And 1,036 of these have made more than 70,137 different code contributions since the OpenStack project was launched three years ago.
Not everyone stays active in any community forever, of course, and the OpenStack Foundation estimates that there are around 240 or so active contributors to the code base in any given month and that over the course of a release cycle there are around 800 to 900 active contributors. This, by any measure, is a large-scale software project.
"OpenStack has a diversity of use and a diversity of community," says McKenty, and this is its strength. "And the fact that OpenStack is past 10,000 members and that it has more than 1,000 active code contributors is astounding.
"More than 10 per cent of community members are actually successfully landing contributions to the code. If you compare that to Mozilla and the work on Firefox, which I was also part of, there were in the low hundreds of people who landed code on Firefox, and their community was hundreds of thousands of people. So the ratio for OpenStack is just amazing."
In the past three years, OpenStack has been beefed up from its initial Nova compute controller from NASA and Swift object storage controller from Rackspace to have additional modules to add network virtualization, bare metal provisioning, virtual machine image management, block storage, access control, and a graphical management console, just to name a few.
More modules will be added as features roll into existing modules. At the moment, the official OpenStack collection is comprised of just under 1.3 million lines of code, and as you can see, the code base has grown fast as features and modules have been added:
Yes, they count blanks and comments as well as lines of code
This chart above shows how the code base for OpenStack has grown over time as each successive release has been outlined and then coded, but what it does not show is the code churn, which is substantial.
For instance, in the last 12 months, the OpenStack project has added 2,936,791 lines of code, but has removed 1,594,506 lines of code. This is still software that is rapidly changing, even as the current "Grizzly" release, announced in April, is arguably the first production-grade implementation of OpenStack that bleeding edge companies can think of deploying in their data centers to build a private cloud or a public cloud (if they are service providers).
It is hard to say if the code base will settle down. As long as the code is improving and OpenStack doesn't break API compatibility, most people won't care or even notice.
"Ultimately, we focused on creating an environment where everybody who wanted to come pitch in and work on this could have an impact, could find a place, and have an effect on OpenStack and therefore on the future of computing," says Bryce.
El Reg tried to goad Bryce into saying even if they are VMware with much taunting, but the most we could get Bryce to say was "even if they are a large, proprietary software company."
"The other thing that is great about this is that OpenStack has forced a number of companies to take a more open approach in how they do development and licensing and how they interact with partners," Bryce continues.
"Again, if you go back three years, everything was not using an Apache license and there were a lot of open-core projects out there. OpenStack was started by users, not by software companies who were trying to do this balancing act of being a software company and making money off some of the software and giving away other parts. By not having that built-in conflict of interest, it freed us up to try to make OpenStack the most open community it could possibly be."