Original URL: http://www.theregister.co.uk/2010/06/15/data_rescue/

Can the cloud hold off data deluge?

There really is a lot of it about... and it all has to go somewhere

By Dan Olds, OrionX

Posted in HPC, 15th June 2010 09:27 GMT

An article in CIO Magazine takes a look at a 2007 report from IDC (sponsored by EMC) estimating that the size of all digital data will grow by something like 1.2 million petabytes from 2009 to 2010, and will grow an astounding 44x by 2020; the number of individual files will increase by 67x.

Even though the report is sponsored by a major storage vendor, the methodology looks pretty solid, and it’s hard to argue against their results. And 44x growth is a lot of growth – even though a significant percentage of that will be multiple copies of my Outlook.pst file scattered across various systems.

The author echoes IDC and makes the point that most of his clients are significantly underestimating future growth, and thus their IT needs. He goes on to discuss the IT implications and makes a case that cloud adoption will have to be significantly higher than supposed, because organizations simply can’t afford to add the storage and personnel necessary to keep pace.

He believes that external clouds will host more than 15 per cent of all apps, which could be accurate; but it really depends on how you’re defining an app. If we’re talking about applications aimed at consumers and casual users – sure, I’ll buy the idea that much of this will be sitting on a public cloud somewhere. But if we’re talking about enterprise workloads, I’m not so certain.

While I agree that the eventual number may be greater than 15 per cent, I don’t think it will be much greater. Concerns about security risks, availability, and lack of predictable performance will trump the potential TCO benefits.

I also believe that most data centers of a reasonably large size can implement private cloud infrastructures and get to cost metrics that are in the same ballpark as the cloud providers. But it’s an interesting topic, and worth a read. ®