The last storage array you will buy
Shhh. Don't tell the vendors
If you are currently in the market for a new storage array, take a moment to reflect on the significance. Your new storage array is probably the last one you buy in the form that we currently think of as 'a storage array'.
The turning point, of course, is flash memory, but that technology is just so 'one dot oh' at the moment. And in many ways it is a problem child in its own right. The significant factor here is the move towards semiconductor-based storage in general, and I'm not entirely sure that the storage vendors are going to know what's hit them.
Semiconductor technology brings the implications of Moore's Law (the number of transistors on a single chip can be readily doubled every two years). One aspect of Moore's Law is the potential doubling of capacity every 24 months, but the reality is that flash technology looks like it is riding an accelerated version of Moore's prediction, and that soon flash storage will hit pricing parity with hard drive storage.
That's all well and good, and of course the storage array vendors will simply replace all their hard drives with flash drives in a rather unimaginative way. But wait: There are further implications to living in Gordon Moore's house and these will be the ones that trigger a full-scale bloodbath in the storage market.
The process of doubling transistor count implies reducing the size of each transistor in the fabrication process, which implies lower power consumption, which in turn implies potential for clock rate increases. This means that Moore's Law brings higher capacity, lower power consumption, AND higher performance on each new iteration of product. Now what happens when that unstoppable Behemoth meets the lumbering giant of the storage industry, with its 5-10 year product cycles and traditional revenue streams?
Supplementing current storage arrays with semiconductor-based storage technologies is a stop-gap measure. Adding flash as a large front-end cache, or as a drop-in disk replacement, is the proverbial lipstick on a pig. That might be a very fine colour you are wearing, sir, but it does not make you Milla Jovovich.
Interconnects are too slow
The answer has to be much more fundamental than simply supplementing existing architectures. The game has changed forever, and everything must be re-evaluated. Ultimately, this is not just a drop-in replacement of any kind - it is a paradigm shift that must span both the software and hardware domains to make the best use of the technology.
Interconnects must be re-evaluated because they are not fast enough, and decisions about what and when data should be cached should be pondered. Ideas that made sense when storage was 200,000 times more latent than memory may not apply any longer when storage is 2,000 (Violin Memory 3200) or even a mere 200 (TMS RamSan 440) times slower.
So how do the storage vendors stay in the game? EMC's acquisition of Greenplum shows the right kind of thinking, but it's hardly a mainstream RDBMS. My money's on Oracle Exadata for the time being - they have all the right pieces on the board and are in the best place to make change where it needs to be made.
James Morle is the managing director of Scale Abilities Ltd. ®