Just how solid is cloud storage in 2014
Airy-fairy vapour or something firmer?
Searching the Reg' for "Cloud + Storage" mentions in 2013 gets you 44 pages of results. So let's take it as read that it's a hot topic.
We're not bothering with private clouds here; they're enterprise IT re-branded, re-tooled and re-priced but they are still private enterprise IT resources. No, it's the public cloud, accessed remotely and made available through a cloud services provider (CSP) that we're considering in this review.
This area is dominated by the AGA threesome of Amazon, Google and (Microsoft) Azure which each have large consumer businesses buttressing their enterprise cloud efforts. And Amazon is top dog, with every other supplier dancing to its tune and reacting to what it does in terms of price cuts and service developments.
The AGA trio have the incredibly deep wallets needed to build out a public enterprise cloud.
Apple has its own cash well to build out its iCloud consumer cloud offering. But this, like Facebook and Twitter, can be classed as an application-environment cloud and not as a general purpose public cloud with storage services. That's not iCloud's prime intent, it being more a repository for Apple users and Apple data, like music and video and photos.
Generally CSPs offer unstructured data storage, backup and archive in the cloud. A few CSPs are offering primary data storage in the cloud. For example, CloudSigma started up an all-SSD-based storage facility for its data centre in the cloud in April. But having primary data in the cloud on flash when compute is across a network link doesn't make sense. Unless you have a genius cloud storage gateway or an expensive high-speed network link you'll learn all about network latency and data access delays.
One answer is to put compute in the cloud too - we're not going to cover that here since it's about IT in the cloud and we just want to look at storage.
The market suffered some significant reliability failures in the year but this didn't stop vendors adding features to their offerings and developing them. They saw that customers were beginning to trust the cloud more and more, and moving data to it; in trickles no doubt but suppliers hoped that this is a one-way street with the trickles becoming streams and then rivers.
Although there were problems with cloud reliability and CSP stability these were not strong enough to derail or even significantly slow the adoption of cloud storage. In fact the main supplier problem was: "How do I compete with and/or differentiate myself from Amazon?" No-one has been able to answer that satisfactorily because the Amazon beast is still growing and defining itself.
Only Google and Microsoft realistically have the heft to take it on directly. Everyone else will have to get niched in the market sense somehow to survive.
Lets start with cloud reliability.
It was not good. February saw the Azure Storage facility go down across the globe because some Microsofty forgot to renew a security certificate. The resulting Blue Sky of Death problems took twelve hours to fix and Redmond said it would offer compensation to affected users as per their SLAs. This surely weakened its credentials as an enterprise user-class storage cloud.
Our Reg writer opined: "It is the opinion of The Register that to have a core service fail in every data center across the world simultaneously is an extremely bad thing to happen to a cloud provider."
The UK's 2e2 cloud service collapsed in February with users asked to pay up to $40,000 extra to get their data back.
April saw Google's Drive cloud storage service as well as its Gmail offering go tits-up for a while.
Microsoft's SkyDrive went down for a while in August, along with Outlook and the SQL service in Azure. Hotmail and Messenger were also affected. Bezos blushed that same month as Amazon's cloud also went down with its Elastic Block Store fingered as the culprit. This was its third such outage in two years.
Nirvanix fell out of the cloud and crashed and burned in September 2013, dealing a thunderous blow to ideas of cloud storage reliability and trustworthiness. Sales didn't grow quickly enough as it built out its storage infrastructure on the back of its start-up funding rounds. So, its backers pulled the plug, with the threat of data lost and unavailable. In the event third-party data rescue efforts recovered, it is thought, much of the data.
September saw Amazon's cloud go down with faults in the company's Northern Virginia US-EAST-1 data center cluster. This affected the EC2 compute service, load balancers, the Redshift data warehouse, the relational database service, and the simple email service.
We wrote: "The Northern Virginia data center is Amazon's oldest public-cloud facility, and has had numerous problems ranging from Elastic Block Store cockups, a huge generator failure, and even a massive general outage in Summer 2012."
Azure fell over in October.
All-in-all the top cloud services did not demonstrate rock-solid reliability in the year.
A constant theme in 2013 was the cost of CSP storage offerings with Amazon pretty much leading the way with a stream of price cuts throughout the year. We speculate that these cuts are to stress out the competition so they withdraw - think Symantec getting out of cloud backup - and to overcome resistance from enterprises to using the cloud. Once Amazon is the market leader in public cloud storage, with virtually all enterprises using cloud storage, then the Bezos behemoth could start raising prices.
It's going to difficult to move petabytes of data from one CSP to another and you could find your self locked-in to the amazing Amazonian accumulator of data.
There were many price cuts during the year. For example:
- Amazon cut DynamoDB prices in March, including indexed storage prices
- In May Amazon slashed DynamoDB NoSQL datastore prices again; for things such as big reads prices dropped up to 75 per cent
- Google cut App Engine NoSQL storage prices 25 per cent in May
Cloud storage events
There were a large number of cloud storage announcements in the year. We've collated lots of them in the list below. Don't bother reading every item; just scan the list and marvel at the amount of activity and the fact that it's not an exhaustive list - cloud storage is HOT:
- In January startup Axcient received $20m in a fourth round of funding to help it develop its agentless data centre appliance which sends backed up data to its cloud.
- In the same month Cisco and NetApp started developing massive capacity FlexPod reference systems for CSPs.
- January also saw Verizon starting up a Terremark brand Enterprise Cloud data centre in in London UK for financial users. It complemented the existing Amsterdam data centre.
- Oracle bought Nimbula and its OpenStack cloud management system in March.
- April is not the cruellest month:
- Amazon added Dropbox-style file sync 'n share to its Cloud Drive. It cut S3 storage prices in the same month and said it was storing two trillion objects in S3.
- Amazon enabled its storage gateway to suck up Microsoft Hyper-V server data to its cloud, building on its VMware server base.
- DropBox added a single sign-on facility, improving its enterprise usability.
- Egnyte bulked up its enterprise file sync 'n share credentials with new execs and board members.
- May saw announcements too:
- HDS introduced its Hitachi Cloud Services, with it being an Hitachi Data Services (HDS) owned and managed, enterprise-level, public and enterprise-to-cloud offering that allows enterprises to store their unstructured data in a remote HDS storage vault, which is built on the Hitachi Content Platform, an object storage product technology.
- May saw Google link its different cloud storage silos into a single unified storage pool. Previously Google cloud apps like Gmail, Google Drive and Google+ Photos had their own storage resource.
- NetApp and Verizon Enterprise Solutions announced a collaboration to deliver Data ONTAP as a virtual storage appliance (VSA) for Verizon Cloud clients.
- June was frantically busy for CSPs:
- HP expanded on its public cloud compute and storage ideas in June, saying it would focus on high-value app niches rather than engaging in full frontal competition with the AGA trio, perhaps using its own kit like Moonshot servers.
- Red Hat announced its own implementation of the OpenStack cloud controller, joining virtually every private cloud and public CSP wannabee supplier.
- Rackspace opened up an Australia base for its OpenStack-powered elastic cloud offering.
- Tier 3 said it had a Riak distributed datastore-powered cloud object storage service.
- Google enabled data upload to its cloud via hard disk; HP, Amazon and RackSpace already offered that capability.
- Joyent started up a ZFS-based object store, the Manta Storage Service.
- SanDisk invested in cloud storage gateway startup Panzura.
- Quantum, known for on-pre tape and disk-based storage products, announced it would start up a cloud-based, managed service provider offering in July.
- In the trivia department Microsoft lost the rights to call its cloud storage service SkyDrive in July; BSkyB already had that trademark. The Microsoft service is now called OneDrive. What a piece of sloppy trademark research by Redmond.
- August saw another burst of activity:
- Quantum's Q-Cloud service for backup, restore and disaster recovery was announced.
- Riak cosied up to OpenStack in August, with OpenStack API support amongst other things.
- SGI backed away from margin cut-throat business of supplying products to CSPs in August. CSPs need low, low cost compute, storage and networking infrastructure and SGI can't provide it at the prices needed and make a profit.
- VMware announced its vCHS virtual hybrid cloud service that VMware said has better than Amazon virtual machine migration and coming backup and on-premise to cloud migration for ESXi users.
- China CSP Tencent offers 10TB of free cloud storage to its customers. Its rivals Baidu and Qihoo 360 offer 1TB. Google Drive offers 15GB, Dropbox offers up to 18GB and Microsoft's SkyDrive 7GB; cheapskates one and all :-)
- Microsoft bumped up SkyDrive Pro (file sync 'n share) service's free storage to 25GB in August.
- September saw more announcements:
- Microsoft offered a dedicated storage cache - Windows Azure Cache Service - to Azure developers in September which competes with Amazon's ElastiCache. Amazon updated its ElastiCache offering the very next day.
- It also said in September it would give away its StorSimple cloud storage gateway arrays to customers willing to spend lots on Azure services.
- And it partnered with Oracle. More on this later.
- Riverbed refreshed its WhiteWater cloud storage gateway.
- The pace hardly slackened in October:
- Verizon started talking up and developing its new public cloud offering in October.
- Nasuni updated its gateway offering then too.
- Microsoft did a deal with EVault to offer a year's free backup to some Azure Enterprise users.
- And in November:
- Symantec withdrew from cloud backup in November.
- Google provided two mobile development tools to developers to persuade them to use its cloud.
- Avere added AWS Glacier support to its NAS accelerating cloud gateway.
- CA can back up data to Amazon's cloud via Riverbed's gateway.
- December saw no let-up:
- Amazon said it would supply its own cloud storage gateway hardware.
- IBM announced a toolkit to move data between clouds.
- Microsoft announced a new Azure data centre in S America and said it had more regions than Amazon.
- Seagate sub EVault announced a faster-than-Glacier cloud archival service.
Let's have a quick check on the three main players, the AGA group.
The AGA players - Amazon
Amazon gained massive additional credibility as a cloud provider in general by winning a large - $600m - CIA cloud project in the face of strong IBM opposition. This credibility edged higher again when Dutch regulators said financial institutions in Holland could use AWS at the end of July.
It increased its edge locations around the world to 42 in July by opening two new ones in India, in Chennai and Mumbai.
It's obvious that Amazon is the biggest CSP and also the most determined to grow. In September its AWS boss and founder, Andy Jassy, criticised private clouds for inadequacy, as a futile attempt by legacy IT vendors to build sand-bagged walls trying to hold back the rising tide of the public cloud. Like King Canute's feeble attempt theirs too will fail.
Jassy said: "To build a private cloud from scratch now is ill-advised. It's not cost effective, it's arcane and it's not the way the world is moving." He said AWS would be developed to appeal more to enterprises.
We wrote, "Jassy did not provide details but promised more features to connect existing data centres to Amazon servers and manage enterprise workloads across them as one. 'We know many enterprise have data centres on premises, and they want a way to use the on-premises data centre footprint with AWS... that's we are doing and are providing.' "
He highlighted these AWS services for big customer appeal:
- RedShift, the data-warehouse-as-a-service released in February, which Jassy reckoned was the fastest-growing service in the history of AWS.
- Glacier to archive and back up data
- CloudHSM to manage data encryption certificates and keys
All the legacy players rushing to the cloud look like the AWS of 2008; they are that far behind, and Amazon said the current state of AWS was only the beginning.
The Cloud dream - data up in the cloud for ever.
Google has a great consumer cloud opportunity with its Chromebooks and Android-powered mobile devices but does it understand web-scale, enterprise IT as well as Amazon which has honed its expertise with years of running its retail operations?
Our impression is that Amazon is more ferociously focussed on AWS and business customers than Google is on its cloud, with corporate focus diluted by Google Glass, driverless cars and the various Chrome and Android initiatives. However Google is determined, ingenious, and can afford to be a strong Amazon competitor.
Microsoft and Azure
Microsoft spent the year playing catch up with Amazon and trying to smother it in the same sort of way it smothered Lotus, Word Perfect and Netscape in the past. It's playing a long game but, almost for the first time in its history, it's met an opponent that may already be too big and too nimble to be knocked down.
So it's looking to partners for help.
In September Microsoft and Oracle agreed to play ball to try and stop AWS or slow its runaway train down.
With the agreement developers can provision the Oracle Database, Java Platform Standard Edition, onto Windows Server, and Oracle WebLogic server onto Windows virtual machines running on Azure. But Linux was included too, with Microsoft offering bring-your-own-license Oracle Linux VMs running Oracle Database and Oracle WebLogic.
This is still catch-up. We wrote: "Oracle's technologies are also available on the Amazon Web Services cloud, though licensing is not wrapped in and the support is less broad. Amazon has, however, been running Oracle tech for several years."
Redmond has an almighty job to do to get on a level playing field with Amazon, let alone accelerate past it.
So where are we?
We can say that nothing fundamental in the technology sense happened to cloud storage in 2013. But the move of backup and archive data to the cloud, as well as the move of unstructured data to the cloud seems unstoppable. It’s reminiscent of the early days of backup-to-disk when deduplication arrived and an irreversible move from on-premises tape to on-premises disk backup started.
It is very early days still. Gartner said in September no public cloud was truly enterprise class in its view. Amazon was closest to that target, with Azure second and Rackspace third.
Despite cloud outages and cloud supplier failures cloud storage as a market grew strongly in 2013. It was helped by cloud storage functionality extensions - like Splunk offering analytics as a service using Amazon's cloud - and price-cutting and, the largest factor, intense competition between Amazon, Google and Azure, with Rackspace joining in too and, latterly, Verizon.
After Larry Ellison's sniffy dismissal of cloud computing some time ago Oracle is now jumping tight into the cloud idea.
We think Oracle’s strategy is to sell Oracle application services to its customers in its public cloud and not be a generic cloud compute and storage supplier. Effectively it’s an Oracle app-only cloud with Oracle not wanting its customers to go to Amazon or other public cloud providers to run Oracle software.
It’s main cloud enemy is Amazon and it’s partnering with Microsoft to help that struggle.
The AGA trio dominate the cloud storage landscape and Amazon, with Bezos as the Aga Khan so to speak, dominates this threesome and everyone else. Can anyone catch it?
The cloud storage market has only been around for a very few years and the total addressable market is huge. It is far too early to call out a winner here. Amazon is in the lead now but who knows what might happen?
Microsoft has a new CEO, Satya Nadella, and the cloud is a major focus for him. Google has an almost bottomless purse and the cloud is critical to it. All three think they can build a significant and enduring cloud presence, which means good prices and good service for customers as reliability and security issues get addressed.
Look for cloud storage to boom in 2014, with Amazon setting the pace and both Google and Microsoft following very close behind. ®