Hybrid Cloud: The new IT service platform?

So. Hybrid cloud. Let's start with a quick definition, courtesy in this case of TechTarget which describes it as: “a cloud computing environment which uses a mixture of on-premises, private cloud and third-party, public cloud services with orchestration between the two platforms”. I like this particular definition as it sums it up nicely: note that by “private cloud” we mean an on-premise virtualised server and storage setup.

It's not all in the cloud ...

How many of us are so averse to on-premise computing that we're determined to move everything out to the cloud? If we consider the desktop as an “on-premise” computing element then next to none of us: frankly I have a very hard time understanding why one might want to replace every desktop computer with a virtual desktop that sits somewhere in the cloud. But even if we're excluding the desktop from the equation, it's still a stretch moving everything to the cloud.

… but it's not all in-house either.

If we look in the other direction, how many of us are so averse to cloud computing that we keep absolutely everything in-house? And I mean everything. Would you have an in-house DDoS protection system, for instance? Surely it's better for DDoS to sit outside your network, as your ISP's upstream pipe is likely to be fatter than yours and hence less easily saturated. Similarly, spam and malware protection for inbound email: there's a lot to be said for buying an external service so that malware never gets near your own systems.

Let's face it, it'll be a mix

These examples could be considered infrastructure utilities, though, and not applications. So what about the apps? Well, it's becoming increasingly attractive to host corporate email and HR systems in the cloud: to run them internally needs much kit, storage and effort. Similarly there are times (perhaps when an app requires direct connection to an in-house system or runs on an unusual – i.e. non-Intel/Linux/Windows platform) when putting stuff in the cloud isn't possible. Finally, disaster recovery (DR) in the cloud is increasingly popular – it's usually a whole lot cheaper than paying for on-premise space in a second data centre.

Given that in the average case we're likely to wind up with a hybrid cloud setup, then, what matters is that we consider the various items of glue that stick the components together.

It's an “it”

The first essential consideration for any hybrid cloud setup is that you think and act as if it's a single entity. So to the absolute maximum extent possible you should avoid thinking of “the on-premise elements” and “the cloud elements” and think of it as a single, integrated system that spans both worlds. This doesn't mean that you'll ever get to the stage where the setup looks completely like a single entity, but you should try to get as close as you can.

Directory services

The first step is authentication, orchestration and permissions control. This should underpin everything you do in any IT setup, and the hybrid cloud is no exception. There's no point running up a bunch of separate applications (particularly those that live with cloud providers) and then trying to retro-fit a central mechanism for them to authenticate connecting users.

The approach you take doesn't really matter all that much as long as you make it suitably secure. One option is to put a read-only directory service in your on-premise DMZ and point cloud-based systems at it; perhaps simpler, as it avoids the need to configure inbound NAT and permissions on your firewall, is to host the directory in the cloud and point the internal systems at it. Which direction you authenticate in is largely immaterial, so long as you have all your systems pointing at a single source of truth (by which I mean a resilient single source of truth, of course – you're putting all your eggs in a single basket so a replicated, redundant setup is mandatory).

Integration software and appliances

If you try to run up an on-premise setup alongside cloud-based systems you can't just tell the components: “Hey, talk to that system over there”. Perhaps, for instance, your cloud apps are split between providers in the interests of resilience or because you're running a combination SaaS services as well as self-managed apps on an IaaS installation. And perhaps you're able to mount a cloud-based storage volume from an on-premise server but it's too slow or doesn't cope with Internet interruptions.

Instead you need to look around the market for mechanisms that let you connect things together. Sometimes these come as part of the applications you're using (e.g. loads of commercial SaaS applications are inherently able to authenticate against Microsoft's cloud-based directory service). In other cases you'll have to buy something to provide a smooth connection between the on-premise and cloud installations (my favourite example is the growing number of storage virtualisation appliances that make it easy to mount filestores between elements of a hybrid setup and which help performance through caching). Note that although a lot of such solutions are referred to as “appliances” many such appliances are virtual rather than being tin boxes that you bolt into your data centre cabinets.

The other essential component is consideration of how you securely connect the compute areas together; unless you're pretty big and can justify a direct connection into the cloud provider (which, you may be surprised to hear, is supported by many of them) you'll instead be using a series of VPN connections between the endpoints. Again we're back to the concept of considering everything from the start: a properly configured set of VPN connections can be run just like a private-wire wide area network, but only if you apply a coherent, properly architected IP addressing and routing regime. As with directory services, this is way easier to do if you design it before building, because trying to bring together a collection of areas with overlapping address spaces simply becomes a hellish nightmare of vaguely comprehensible address translations that make system management confusing and fault diagnosis hideously tricky.

Management tools

The same applies to management tools at all levels – some of which are easier than others to deal with.

Application level

This is generally the easiest management layer to deal with if you have a truly integrated hybrid world, because you'll point whatever you're using to manage the applications at the various sites and (as described in the previous section) they'll be able to route and connect correctly. So there's no reason that, say, your enterprise malware orchestration engine or your database replication can't run seamlessly across your entire estate regardless of endpoint location.

Operating system level

The same applies at OS level: if the servers and other systems in the various locations can see each other at an IP level, layering the OS management tools and their associated funky bits on top is easy. Even back in the 1990s I remember having a super-slow global network spanning the 18 or so sites over which we were able to layer things like Novell NDS, a global DHCP service and what purported to be a “cluster” technology for the network of voicemail appliances scattered over the world. Co-ordinated IP connectivity, even at a super-basic level, is a great enabler for the OS-level stuff you'll want to use.

Virtual machine level

If the operating systems can talk to each other, you have the basic ingredient for managing your virtual servers in a unified way. The main problem, in fact, is finding a tool that knows how to manage a bunch of different servers. If you have similar systems in the cloud and on premise (Microsoft Azure in the cloud and Windows on-premise) then this isn't terribly hard. For less similar installations it's harder and you don't get the chance of seamless or even near-seamless management – though all is not lost. For instance Amazon's CodeDeploy tool can shunt app updates to both AWS and on-premise servers, and Google's StackDriver management tool can monitor AWS as well as Google's own cloud. Finally, you have the third party vendors like RightScale who have well spotted a gap in the management arena and are doing its best to sell us services to fill that gap.

Getting deeper?

Can you manage universally any further down? No, because in a cloud setup you can't see any lower than the virtual layer anyway. So your tools for controlling your physical hosts' lights-out management adaptors and the low-level config of your networked storage will always remain on-premise only, as will the management interfaces for the devices (such as the storage appliances mentioned earlier) that provide the glue between locations and platforms.

Upgrade co-ordination

There's one final point worthy of note, because it's a core difference between on-premise and cloud installations – primarily when the latter are SaaS platforms, actually. That's the co-ordination and roll-out of system and application updates.

If you're running an IaaS setup in the cloud then co-ordinating system upgrades is pretty straightforward because you'll be responsible for both your on-premise and cloud setups. As you've got the comprehensive, organised network we discussed earlier, whatever update manager you use should be able to distribute patches across the estate as per your defined policy.

If you run some SaaS components, though, you'll have to make an exception and deal with software updates for the self-managed elements to ensure they keep up with the version in the cloud. SaaS vendors will be implementing both OS and application patches based on a schedule, and you need to ensure that you keep up because many apps deprecate features over time and if you fall too far behind you'll find things stop working. It shouldn't be a problem keeping track of SaaS updates as the service providers will publicise them, but you do need to read and act upon the alerts.

Security

We've not mentioned security directly yet, but that's deliberate as we've been building up to making it easy. “Security” as a concept means more than just making sure users change their passwords regularly and that each login ID has access only to the right files and folders: it incorporates the concepts of governance, compliance and risk management.

The point is, though, that a comprehensive hybrid environment that's largely centrally managed is little more troublesome than an entirely on-premise installation with regard to security. Yes, you have to be cautious to protect the crown jewels that is your core authentication and orchestration mechanism (particularly if it's cloud-based) but aside from this, a nicely integrated hybrid world can still log to SIEM services and centralised log servers, with reporting as easy for the external components as it's always been for the internal ones.

Your network and access control list design should, as we've mentioned, be universal and relevant to the entire estate, which ticks the governance box nicely. Risk management is also not made overly more complicated by extending the world outside your own premise because you're not introducing a new set of systems – it's more like you're extending your existing systems to environments that just happen to be located outside your premises. You do have to be mindful when it comes to compliance (not least with concepts such as data export regulations) but since the cloud isn't just a mystical soup of processing but is segregated into known geographies this is a reasonably straightforward box to tick.

Summary

Looking back to the first sentence of this article, we wrote that hybrid cloud is: “a cloud computing environment which uses a mixture of on-premises, private cloud and third-party, public cloud services with orchestration between the two platforms”.

There's one word in that definition that's utterly crucial to a hybrid cloud setup: orchestration. The most important thing with hybrid cloud is that it acts, to the greatest possible extent, as a single entity that just happens to be spread between locations, providers and technologies. At the basic level you can make in-roads to unifying the management of the elements – and you can help yourself achieve this by considering the range of providers and platforms you intend to use before you start to build (let's not run up an Azure cloud and slap in a VMware on-premise infrastructure, for instance, because it's going to cramp your style integration-wise).

Managing the lower layers won't be entirely unified, but the cloud providers and the third party software suppliers will at least give you a leg-up and make it look like something other than a collection of unrelated islands.

But with sensible design and a decent network architecture, everything from the OS upwards can be made to look like a unified infrastructure to the server, OS and application guys – while the governance and compliance team look on happily.