IDC convinced data will crush universe
Short term memory
Comment It's May, and IDC has unveiled a doom and gloom Digital Universe predication on behalf of EMC. It talks of quintillions of data containers, saying we face a perpetual tsunami of digital data which will grow 44 fold from 2009 to 2020 as the number of IT professionals grows a mere 1.4 fold over the same period. Oh, woe is us. Oh, woe is - enough already.
Imagine a report written 40 years ago about the same topic: information - but when information as stored as inky squiggles on paper. Remember that? IDC, the Intelligent Documentation Consultancy, is contracted by a paper production supplier, writing that between 1979 and 1981 the amount of information produced in the world is growing 44 fold, that the number of containers for it is growing 67 fold, because smaller pieces of paper were being produced and needed to be stored in lever arch files and such like. But the number of filing clerks is only growing 1.4 fold. There is a storage gap. Not enough book shelves exist. Oh, woe is us. We will be undone.
This tidal wave of paper is a big problem for companies and, well, you get the picture.
The thing of it is that the majority of paper documents were trivial and short-term: shopping lists, receipts, birthday lists, temporary phone number records, and the like. This stuff wasn't worth keeping and was dumped in the waste paper bin. What IDC fails to understand with its Digital Universe effort is that a vast amount of consumer-generated digital data is crap. Just because we have 5GB of photos and MP3 files on a smart phone doesn't make them precious lifetime memories or essential data to be archived. When storage is essentially free, it gets filled with fluff, rubbish, digital dross.2 The solution to the digital universe is the delete key.
Sure, business can analyse trillions of micro-records about resource usage from smart devices, but this is just self-flagellation. What storage company is ever going to tell actual and prospective customers to simply throw data away or even not collect it in the first place. Certainly not EMC or IBM. It's in the collective storage industry's interest to big up the problem and IDC is gleefully bigging it up.
So what that there isn't enough digital storage to store all the digital data that's going to be created? Why ever should there be? If you think this is a problem you don't understand the issue. There is no problem. It's ever been thus with whatever medium we use for storing information. There's always been more information produced than there's been the capacity to store it. Deal with it. Live with it. That's what people have been doing for hundreds of years. Masses of information is produced transiently and has no long-term use whatsoever.
Just because the data is now stored digitally doesn't alter this. The medium here is not the message. The message is the same whatever the medium used to store it. It's been the same for the Clay Tablet Universe, the Parchment Universe, and so on.
The trouble with IDC is that it only has a short-term memory. Get a sense of history guys and stop producing this Digital Universe tosh. ®