The Great El Reg Private Cloud survey: Results are in!
Applications must work for the cloud to float
We were surprised at the strength of positive feedback from the 570 readers who participated in our recent Reg reader survey on private cloud.
It was quite a contrast to surveys on hosted cloud services, or even cloud computing in general. Whenever we run these, readers tend not to be shy about telling us how much they think all that public cloud stuff is over-egged by all the vested interests, revolutionaries and religious types.
This is useful in some scenarios, perhaps, but not some kind of magic that will let you shut down all your servers and go lie on a beach, as some would have you believe.
The reason that private cloud is being received more favourably by IT pros is that despite the ‘C’-word in the name, it is really just the natural consequence of several trends that have been unfolding in the data centre world for a decade.
In fact, our survey respondents made it clear that they while view private cloud as a natural next step from x86 server virtualisation, they don’t think it necessarily has much to do with public cloud services at all.
What is private cloud?
Here is how we defined private cloud to respondents in the research study:
The basic idea of Private Cloud is to pool a bunch of servers and other resources (storage and networking) to create a general-purpose platform upon which a variety of workload types can be run simultaneously. An important attribute of private cloud is the rapid allocation/de-allocation of resources to/from workloads, enabling a more dynamic approach to management.
We also made it clear that private cloud (at least in our definition, which echoes that of most other sources), is about architecture, not services like public cloud, and is generally applied in your own data centre.
We also pointed out, however, that a private cloud could be set up on dedicated infrastructure in a hosted colocation-style environment.
Some readers commented that our definition had missed the fact that private cloud enables a self-service approach to IT delivery and a charge-back or show-back approach to IT related account and internal billing.
Both fair points, but given the emotive nature of self-service and charge-back, we deliberately left these out as we didn’t want to imply that they are prerequisites for a private cloud installation. They are options for those who find them useful.
More important in terms of benefits are enhancements to flexibility, responsiveness, application availability and overall efficiency. If you want to know more, check out the main report from the research study Private Cloud in Context (reg' req'd) .
Who is using private cloud?
Unfortunately, online surveys can be not much use for figuring out the real level of adoption of an emerging technology. This is because those who are more involved in the topic are more likely to respond and are therefore over-represented.
We can, however, investigate where early adopter activity is taking place in relative terms, and with private cloud at the moment it is among those with larger IT infrastructures (figure 1).
This is understandable. The problems addressed by private cloud are more acute in large-scale IT infrastructures. Also, big IT departments tend to have more experience with virtualisation, which is a natural springboard for private cloud activity.
There is, however, clear evidence that private cloud is not only for the the big guys: successful adoption is also taking place within small and mid-scale environments.
But all is not sweetness and light. Despite the generally positive reaction, respondents in our study were very realistic about adoption practicalities.
Understanding what is involved in terms of migration and coexistence is important. Private cloud simply provides a platform and operating environment, which counts for little unless you can deploy applications on it successfully.
A question of compatibility
Most of the emphasis in private cloud is on building flexible pools of x86 server resources.
It is certainly possible to build other flavours of private clouds, for example based on Power or Sparc architectures, but most of the problems of application and server sprawl that private cloud can help resolve are found in the x86 Windows and Linux world.
For application compatibility, small footprint software that has already shown itself to be virtualisation friendly (in other words it can run well on a hypervisor-hosted operating system) is likely to run on a private cloud too with little or no problem.
Some comments from the respondents in our study highlighted how straightforward things can be.
“I have worked with this stuff day-to-day, and it's very rare that an application needs anything special to work on a private cloud.”
"I have not come across any x86 based app that will not run perfectly in our private cloud.”
We have no doubt that it really is this easy, particularly if you work in an environment where virtualisation is normal, where there is good discipline with regard to application design and selection, and where you don’t have a lot of legacy kicking around.
However, quite a few respondents, including many of those with private cloud experience, had concerns about application compatibility and urged caution (figure 2).
"Apps absolutely must be capable of clustered functionality. None of this 'won't write to centralised storage/single access database/etc'.”
“I'm amazed at how many software products are going back to dongles to limit their installations. This really sucks when trying to virtualise them.”
“Old software that requires a physical box with crazy hardware requirements while barely being able to run under load will just have to go the way of the dinosaurs.”
Many small-footprint applications of the kind you would want to deploy on a flexible shared resource are developed internally, for example to meet the needs of specific departments or workgroups.
This raises the question of whether development practices need to ensure that custom applications are built with cloud readiness in mind. Some respondents did indeed think developers might need to smarten up their act.
“Not only will apps need to change but capabilities of developers will require a shift change. Current popularly used development practices do not scale into the cloud."
"Most of our biz apps are kinda small and naively architected. For cloud deployment, web apps need to ensure they're data-less on the server, with persistence provided by backend DB/NAS so that we can scale them. Many of our devs don't get it.”
This brings into focus that old chestnut: the frequently encountered disjoint between development and operations.
Particularly with run-of-the mill tactical apps, developers should not be making decisions that constrain the freedom of operations staff to deploy and redeploy between different hardware and platform environments as needs dictate.
What about the big stuff?
In larger environments in particular, a private cloud is hardly going to be the first time anyone has deployed a multi-server architecture to underpin an application.
Various forms of specialist farms and clusters have been used for years to support more demanding requirements (figure 3).
A question that will increasingly arise over time is therefore which of these applications are legitimate targets for migration into a private-cloud environment.
Your first reaction might be to ask “why would you even think about migrating them?”, on the basis that many of these apps are business critical and if they are all running fine it is best to leave well alone.
The answer is operational efficiency. Firstly, it is about reducing fragmentation and disjoint, with each specialist platform requiring a separate set of dedicated tools, skills and procedures.
Our research consistently throws this up as a massive killer of productivity and an ongoing source of frustration for IT professionals.
Secondly, the broader the pool of resources across which sharing takes place, the greater the level of utilisation and overall efficiency.
While migration might be desirable, we come back to the basic question of whether specific multi-server applications will operate successfully in a private cloud. Here is what some of our respondents said:
“Distributed processing has been very lightly touched by cloud computing and most applications are not ready for it, much as most applications are poorly prepared for multiple server cores. There is a great deal of room for improvement in the area of parallel and concurrent processing in software development.”
“Application architecture is critical here. Most existing apps would require significant work to migrate to public cloud offerings as they exist today. Less work (but still some) is required for private cloud. If no remediation is done, apps will not be able to leverage scale-out, clustering, etc. It will be no better than just server virtualisation.”
“The big problem for us is multi-threaded vs multi-processing, e.g. with big analysis applications.”
“We're at the 'new services must be virtualised' stage already. Private cloud implies 'all services must be redundant and scalable' also.”
Migration vs integration
In practice, it is unlikely to be an all-or-nothing situation with regard to migration of specialist farm or cluster-based applications.
With one application, for example, the proprietary runtime and management platform might be done away with altogether and all of its functions fulfilled by a private cloud.
You might then have situations in which runtime execution remains dealt with through a dedicated specialist control layer, with the management layer integrating with the private cloud to coordinate the provisioning and release of resources between environments.
Either way, there is a good case for integration to allow coordinated monitoring and event management (figure 4).
With this in mind, views of how some of this will pan out are already forming in the minds of those with experience of both private cloud and the different types of specialist farms and clusters (figure 5):
But being able to substitute elements of your specialist management and control environment with the private cloud equivalent or pull off any serious integration requires action on the part of application and middleware vendors.
The trouble with vendors
Some of the feedback gathered during the research, particularly from those with more experience, expressed frustration with some software vendors.
“Many vendors talk the talk but fail to deliver on promises. In some cases, it's not certain that they actually understand the real concept behind cloud computing. They deliver just a package to run on a web server and think that they have done all that is required.”
“I expect migration of packaged apps, especially from niche players, to be very slow. We will run a two-tier infrastructure for probably a decade while the legacy apps work through.”
"Too many vendors ignore the migration process, the time we are going to spend with part-transition to cloud in place, and how their tools interact [with what’s already in place]. Until we see more real effort on standardisation this isn’t going to get much better."
"Many LoB apps that we support are selected by the end-user without consideration to the ICT backend, so we end up with applications that require legacy operating systems etc. Many of these application providers don't even have a roadmap to get to the latest version of software, or even certify their product on a virtualised platform, so certification as 'cloud-ready' seems a long way off at the moment."
What is needed at the moment is clarity from vendors on what they will or won’t commit to, and perhaps some mechanisms to provide assurances that solutions will work in a compliant manner (figure 6).
Quite how far some of the larger players will go in opening up their environments to take full advantage of generic private clouds remains to be seen.
For some it will be a case of support and maintenance practicalities – everything is easier and you can be more confident of being able to troubleshoot and fix problems when you are in control of the layers.
For others, who seem obsessed at the moment with hardwiring all layers of the stack together in the name of optimisation (aka customer lock-in), there is also a question of motivation.
“Many are wedded to the idea of selling the whole package – h/w, s/w and services.”
But it is not just vendors that may have to change.
All pull together
IT departments, especially larger ones, are often organised into operational silos, with separate teams looking after servers, storage, the network, the desktop, applications, security and so on.
Faced with how disjointed toolsets, skills and processes have become over the years, many are coming to the conclusion that something needs to be done.
Virtualisation has already started to act as a catalyst here. It is increasingly common for teams looking after virtual environments to find they are having to work across domains. To take full advantage of private cloud, this kind of coordination becomes mandatory.
It doesn’t necessarily mean losing the silos altogether: the need for specialist skills doesn’t go away. But it does mean breaking down a lot of barriers and driving towards better integration of tools, policies and processes.
So, while many of you out there are picking up on the potential of private cloud and asking if it is ready for mainstream deployment, a more pertinent question might be whether your organisation is ready to take it on. ®
This is the first article in a two-part series. [The second article is here.]The study upon which this report is based was designed, interpreted and reported by Freeform Dynamics, with data gathered from 570 respondents via an online survey hosted on The Register. The research report "Private Cloud in Context is available for download at The Register's Whitepaper library.
Dale Vile is managing director of Freeform Dynamics, the UK IT analyst firm.
Sponsored: Benefits from the lessons learned in HPC