Psst. Hey. Hey you. We have to whisper this in case the cool kidz hear, but... it's OK to pull your data back from the cloud

Analysis The concept of cloud repatriation – shifting systems back in house from the cloud – is nothing new. For as long as there have been cloud services, there have been those who have hosted applications and workloads off-premises before bringing them back in.

The past few years have served up some high-profile cases: file storage and sharing specialist Dropbox took hundreds of petabytes of customer data off AWS’ and back into its data centers in 2017. The company saved almost US$75m in infrastructure costs over two years according to an S-1 filing.

Craft brewing company New Belgium Brewing this year repatriated a number of critical workloads including business intelligence, ERP and CRM applications, and SQL and SharePoint databases. In doing so, the company estimated it will reduce its operational costs by up to 66 per cent per year, as well as cut production downtime and boost capacity on its bottling line by 25 per cent.

"Our most mission-critical workloads simply must be run locally," said Adam Little, senior systems administrator at New Belgium Brewing at the time. "This includes programmable logic control and process automation for brewing—right down to opening and closing valves. We can't risk running that over a wide area network (WAN) link."

Elsewhere, ecommerce retailer Teezily estimated it saved €400k a year on storage infrastructure when it moved 450TB of data off AWS S3 to a fully self-hosted cluster based on OpenIO SDS.

Figuring out the cost is hard to do

Calculating the true cost of migrating any workload into the cloud, and vice versa, has always been tricky, with cloud service providers often criticized for opaque pricing that makes the long term TCO difficult to tally. Combined costs vary considerably according to the server type, the number of virtual machines and CPU cores, the amount of RAM, and hypervisor, OS and database engine choices, as well as storage type, migration expenses, testing, and consultancy fees.

Accurate comparisons also require a thorough understanding of owning and maintaining on-premises infrastructure – both direct (hardware, software licenses) and indirect (power supplies, connectivity, air-conditioning, staff salaries etc) costs, which can be equally hard to add up.

IT departments can easily spend more than they have budgeted for when storing data in public clouds, often due to the cost of moving high scaling applications out of things like Amazon EC2 and into reserved instances that attract a price premium. So called “lift and shift” migrations that see legacy on-premises applications transferred into public clouds with no re-engineering can also be complex and expensive.

Figures compiled by the Active Archive Alliance calculated the relative cost of storing one petabyte of archival data on a flash storage system as $3.5m, network attached storage (NAS) box $2.6m, and Amazon S3 at $1.5m. Storing the same information on tape on the other hand, added up to just $107k.

That suggests that some organisations are not as hot on archiving data to secondary and tertiary storage platforms as they need to be, and may be paying through the nose to keep information instantly available in the cloud either through poor management or long-term contract lock-in as much as anything else.

Not just costs

While cost reduction is a common theme, it’s not the only reason for the change. In 451 Research’s Voice of the Enterprise (VoTE) Cloud Transformation, Organisational Dynamics 2017 report, 34 per cent of respondents said they had moved workloads of a public cloud to a private cloud and on-prem. Reasons cited were performance and availability issues (47 per cent) with high costs of cloud services pinpointed by 40 per cent, and the need to meet regulatory and data sovereignty requirements by 33 per cent.

IDC in 2018 confirmed the trend but places security concerns at the top of the list. Speaking at the IGEL Disrupt conference last year, its global vice president and general manager and research fellow for data center and cloud Michelle Bailey said 81 per cent of customers surveyed reported the repatriation of workloads from public to on-premises private cloud, hosted private cloud, or on-premises non-cloud infrastructures, a figure expected to rise to 85 per cent in 2019.

Respondents predicted that about 50 per cent of their public cloud applications will move to private cloud or on-premises infrastructures by 2020, with security issues driving about 19 per cent of that move, performance issues 14 per cent, and cost issues 12 per cent, Bailey added.

Secure this

Security concerns have long been cited as a deterrent for cloud migration – one reason why service providers have worked hard to improve their defenses and security management processes in the last few years.

Even so, many organisations – particularly in industries such as financial services and healthcare regulated by stringent governance and data protection rules – remain apprehensive about storing and hosting sensitive information pertaining to customers and patients off-premises. Data residency and sovereignty requirements often lead some to insist data is stored only on infrastructure located within specific geographical legal jurisdiction.

While those organisations tend to be in the minority, others worry that automated cloud applications and workload provisioning undermines their ability to control how sensitive data is transferred and processed.

None of that means cloud hosting infrastructure is actually insecure – most dedicated third-party data centers arguably have stronger physical and digital security in place than their customers and employ staff with better knowledge of local regulatory landscapes that can be safely applied to service provision and hosting policies.

The weak link is more often the network – the transmission of data between the end-user device or on-premises application server and off-premises infrastructure that passes through multiple switch and router hops and is vulnerable to hacking and interception. That and continuing confusion over where the boundaries for responsibility between customer and cloud provider lie in the event of any data breach from a legal perspective is often all the reasoning a chief executive needs to keep data within the organisation’s own firewall.

Most need hybrid balance

If incidents of data repatriation tell us one thing, it is that there is no longer any clear cut right or wrong approach to data hosting. Rather, most organisations will benefit from using a balance of different services and infrastructure to match the requirements of specific workloads and applications that usually differ and can change over time.

Hybrid colocation and the careful allocation of data sets to different clouds, taking into account performance, cost, and security metrics, should avoid problems that force repatriation at a later stage. Equally, organisations need to stay flexible and be prepared to alter the provisioning mix within multi-cloud frameworks that incorporate both on-premises infrastructure and off-premises hosting as and when they need to.

“Yes, there will be workloads that have migrated to cloud that will move back to a traditional or on-premises delivery method,” said Cassandra Mooshian, senior analyst at Technology Business Research. “However, there will also be services deployed on premise that could eventually be moved to a cloud environment as customer needs and costs change.

Big IT suppliers have also worked hard to deliver converged server, network, and storage architectures that make use of lower-cost disk drives and power efficient CPUs to bring down the cost of owning and operating on-premises or private cloud deployments.

The assumption is that enterprise clouds will increasingly mix public cloud with either on- or off-premises private enterprise clouds using container stacks like VMware, Azure Stack, Openshift, or Cloud Foundry, to provide the provisioning, orchestration, and management glue that binds them all together.

Against the trend

Make no mistake, cloud is a force in IT. Forecasts indicate that the public cloud services market will continue to expand, not shrink, in the near term. Gartner predicts that the global market for public cloud services will expand by 17.3 per cent to US$206.2bn this year from US$175bn in 2018, for example. IT departments should, though, remember that cloud is neither inevitable or irreversible. ®