Make life easy for yourself when you move to the cloud

So the boss has decided that the company needs a “highly scalable elastic cloud infrastructure” and in true Dilbert style, the rank-and-file IT people have to build said cloud yesterday.

The most important first step is: don't do anything... Yet.

An organisation’s, or at least the IT department’s, staff need to understand not only how it is going to be moving its stuff into the cloud but also why. And “because I read about it in an inflight magazine” is not a good enough answer.

There has to be a compelling reason for the move rather than just CEO buzzword bingo.

What is the company hoping to achieve by moving to the cloud: reducing costs through removing your internal infrastructure and cutting power and cooling costs? Or is it the belief that cloud allows a company to be more adaptive and agile in managing infrastructure demand?

Elastic fantastic

As the name implies, elastic compute expands and contracts on demand and the business pays only for what it consumes. An example of cloud done correctly is Airbnb, which uses Amazon Web Services to bring on lots of additional temporary capacity at peak times to cope with the influx of buyers and then remove the nodes after the rush.

While this solution may sound appealing, superior trouble-free elastic scalability does not come easily. Obtaining this level of automation requires skill, knowledge and planning, as well as extensive testing.

Cloud is not just a case of shifting everything onto another company’s tin. It is a paradigm shift in the way you and your whole company should work. The world of the sysadmin has moved beyond simple virtualisation platforms and associated management tools.

The next stage in cloud evolution is extensive automation of processes such as virtual guest creation, lifecycle management and eventual destruction. In such an environment, standardisation and automation are your allies.

Building infrastructure for elastic cloud takes a whole new skill set. It doesn’t just differ from what you could call classic IT; the interactions between administrators and other users and groups are also very different.

A core principle of elasticity is that individual virtual machines are no longer important for the most part. Rather than trying to fix a broken guest, just destroy and redeploy. It is quicker and cheaper than trying to fix the broken virtual machine.

This scenario is often likened to cattle versus pets. When pets are sick, you care for them and nurse them them back to health. If a member of a cattle herd is too sickly to be saved, it gets shot.

A matter of life and death

The bigger picture means that no virtual machine should hold stateful (important) data. All the data should be backed off to persistent storage or preferably a database. This stateless setup is the bit that enables you to deploy additional virtual machines.

The unimportance of single machines and the automation of deployment and sizing is the key principle. Your web developers need to get to grips with how elastic works and how data needs to be managed in an environment where machines live and die from moment to moment.

From previous experience I know that developers have a nasty habit of scaling up rather than down, just throwing more and more resources at fewer machines. This goes against the whole ethos of cloud.

Also doing it this way you will reach a point where the machines cannot easily be extended further.

Managing the costs of creating virtual machines gets simplified and replaced with an approval process to control the costs. Someone at the end of the day has to pay the bill. Put bluntly, unused or powered-on machines in the cloud will cost you real money.

Herding developers is like herding cats. They may well demand several new virtual machines to test their applications but the cost is not zero. For this reason, each group of virtual machine creators should have a business approver whose job is to balance the resources and costs against the requests.

The business approver be able to modify requests for huge resources

Frequently developers may ask for a new dev environment that is huge, just because the vendor says it is the recommended spec.

The business approver should not only be able to approve or decline new servers, but also to modify requests for huge resources down to a more reasonable level and have the automated solution then go off and build that virtual machine, or group of them.

This leads to a bit of non-IT management: the service catalogue. Any reasonably sized estate should have one.

In essence it provides a menu of services and functions that the company provides. The only difference is that the chef shouldn't go and make it custom. Ad hoc individual customisation is the nemesis of cloud.

Larger companies will have in-house engineering teams who develop the cloud offering, but smaller companies have to rely on their admins’ intuition and know how to develop a good offering. Getting this part right is critical, especially if there is a desire for charge back or show back.

Incredible shrinking cloud

Cloud automation infrastructure should make your life easier. If your cloud provider doesn't have good management and automation tools, you have an issue.

Cloud elasticity is important. Any company worth its salt knows that the ability to scale up and scale down drives down costs. This requires massive automation.

There may be smaller companies that want one or two machines, and that’s fine, but for the rest of us, getting the automation and deployment is key to avoiding tears later.

Also moving to the cloud can hopefully fix those mistakes that have lingered in your infrastructure since the deep distant past.

These types of decisions are just the start. Although all the facts and information we present here applies equally to all types of clouds you still need to decide what type of cloud you want to use for your environment.

Do you want public, private, hybrid or on-premises? These questions need to be addressed up front as they will have a fundamental effect on how you design your cloud.

One perhaps strange thing that you may find yourself doing is defending your choice of cloud. Several recent migrations I have been involved with have had a number of armchair IT gurus telling us that cloud platform x or y is cheaper than the platform we have chosen.

The reply to such people, should be: “Our cloud provider was chosen to provide a standard platform for the company and this is the only platform that the company has committed to support.” Be warned: this will crop up several times.

Moving to cloud is also potentially a massive jump for the developers. Most companies have a defined set of applications that they deliver to their customers, both internally and externally.

Automation comes to save you from this old-school solution

Typically, setting up servers has been very labour intensive because several teams were involved and each applies its own layer of secret sauce. In the new cloud world this just doesn't cut the mustard.

Automation comes to save you from this old-school solution. Most major cloud providers provide automation solutions for free, as well as paid-for solutions from third parties.

In terms of nuts and bolts, you may well find that things are changing. The new technologies that you may have been reading about, such as Puppet and Chef as well as several other orchestrator products, may well be key to what you are trying to achieve.

Putting some time into learning these tools is a very wise investment, as the future of administering in general is efficiency: do it once, script it, test it and deploy it.

Risky business

Management of the security infrastructure is vital. With cloud, essentially you are putting a lot, if not all, of your eggs in one basket. Managing security and access becomes key.

Ideally in a modern environment with several groups you will have self-contained groups or companies and a management infrastructure so that you can manage the towers or groups that are in the cloud.

A good authentication infrastructure, well used, is the cornerstone of any cloud. I would argue that sound data security practices are more important in the cloud than they are on your own infrastructure, so give it serious thought.

Yet another facet of management is what goes into the tools and scripts that manage the environment. Yes, version control – a bit of a Marmite scenario. These tools are key. At some point a script will need to be rolled back: how do you manage that ?

Developers are very much the same. Cloud deployment lays the fundamentals for automated deployment of new versions of software but this should be managed and deployed in a controlled manner.

Essentially, you need to get the developers on side and show them how all this elastic automation means an easier life for them.

Cutting corners just stores pain for later. You shouldn't need to cut corners as a fully automated deployment process should be as quick as it needs to be. Did you say you had a manual process somewhere? It could fail.

When you no longer manage your own tin, you have to make up for the lack of easy oversight. There are a couple of steps you need to take to ensure you don't end up with issues or unexpected bills.

Most fundamentally you need to set up and configure logging in the environment so you know as soon as you have issues. I would also suggest that while you get used to your environment it would be wise to turn logging up and then adjust it as needed. Too much logging is easier to deal with than “Er, I don't know because logging was off.”

On the money

My last tip for running an effective and efficient cloud environment is based on the cost of the cloud. Firstly, having warnings when your cloud hits certain monetary values means you will not get a shock when the bill arrives. You can also adjust resource usage if needs be to cut the cost.

On demand is the most expensive way to run a cloud. The clever money buys resource up front. Purchasing reservations of compute and the like up front can save large amounts of cash. Working out how much you need day to day represents the most effective use of your money.

As the old adage goes, own the base, rent the spike. In other words, know your costs for the month and bring on additional capacity, the spike, as and when you need to.

In summary, moving to cloud enables a company to leverage the agility of easy-on, easy-off excess capacity, unlike the old on-site farms where bringing online resources for a short time can be expensive and limited.

A lot of people are worried about the cost of cloud, but done efficiently and with a well thought out plan, there is no reason why any modern company, be it startup or major corporation, should not be looking to take advantage of the cloud environment. ®

Find out about the largest Amazon Web Services public sector cloud event in the UK here