Are you in the 1%? The 1% of sysadmins who need specialized flash?

Opinion Last week at Tech Field Day during VMworld, I met Violin Memory. A company that has been a pioneer in flash storage, but now struggles because, I believe, it has lost some of its credibility due to the lack of a decent feature set.

I'm thinking of data services, integration with hypervisors and OSs, and so on. They are catching up on features with the Concerto 7000 family, but, looking at their revenues too, they are still in trouble.

Perhaps this is more of a marketing problem than a real technical one, but now Violin has to maintain both hardware and software research and development, which adds cost, less flexibility, and longer product development cycles.

For example, some startups in this space have already released products based on TLC and 3D NAND, lowering prices and now competing at around $1/GB. When will Violin be able to do that? El Reg's Chris Mellor, in one of his recent articles, is wondering if proprietary/specialized design is better than commodity.

I think that the answer, as often happens, is "it depends." And I think it mostly depends on your target market.

Flash is fast, durable, efficient

Coming from traditional HDDs, flash is faster. A single enterprise-grade 2.5-inch SSD can sustain 60/70k IOPS with a very good latency. And now, thanks to NVMe, we will get even better results both in terms of IOPS and latency.

In fact, the major problem we have today is the SAS backend, not because of SAS itself but because of the many layers of controllers. And, yes, also partially due to a SAS protocol that was not designed for flash. Capacity looks like a problem of the past too. 4TB drives are now available and their power consumption, still in the 7 watt range, is also improving the overall power efficiency of storage systems.

And you know what? Flash is more durable than disk, too. Failures are much more predictable, flash drive controllers are much more efficient than in the past, and array vendors know how to work around NAND memory limitations and constraints. So, from the end user standpoint, you have comparable prices, strong perfomance, and better efficiency than what you had in the past. The net result is that end users can now buy flash arrays at disk array prices – 10 to 100 times faster – at the same price and with better features. Do we really need more today?

Someone needs more

99 per cent of workloads can be solved with general purpose arrays. But in some cases, you are in that 1 per cent and there isn't commodity hardware in that space. Economically speaking, for most vendors, it simply doesn't make sense to produce denser, more efficient and faster hardware: they can't sell enough of it to justify R&D and production costs.

But it is also true that if money is not a problem and you want more performance and efficiency, the only solution is specialized hardware. Companies like EMC, DSSD – and, once, Violin – target these markets (and they are in that 1 per cent): high-speed financial trading, some big data applications and very high-end transactional DBs just give you results that much faster if you give them more resources.

This is also true if you need the highest possible capacity and power efficiency. You can't pack half a petabyte of flash in 3U with commodity hardware. But how many people need it today?