For EMC, it’s all about the rack-scale flash
Are we seeing an embryonic disk SAN killer tech?
Comment EMC has two big engineering projects on the go, incorporating flash-based DSSD and virtual SAN ScaleIO, both looking to provide rack-scale storage.
DSSD is an acquisition EMC made in May last year, and is working on a design using PCIe-accessed flash as the persistent storage tier.
The idea is to have racks full of flash for low-latency access to in-memory databases and real-time analytics workloads.
How much flash? A 33U XtremIO 6-X-Brick cluster has 120TB of raw flash. Calculating out the TB/U and deriving a 42U rack's worth from that gives us 151TB. With HGST's Skyera gen 2 Skyhawk product offering 136TB in 1U the extended XtremIO number looks far too small to make much of a fuss about. A 42U SkyHawk rack would offer 5,712TB – now that would be something to write home about.
Surely DSSD is looking to produce petabyte-plus rack capacity. At a Stifel Tech conference EMC's Chirantan “CJ” Desai, president of its Emerging Technologies Division, said that the DSSD product would launch later this year and have both block access and HDFS support. He said it would not use off-the-shelf PCIe flash cards but use PCIe as the channel to access raw NAND flash.
This implies it doesn't use DSSD either. Instead, like Violin Memory and HGST's SkyHawk, it is directly addressing NAND dies and managing write reduction, garbage collection, and so forth.
The DSSD product would be denser than the XtremIO array. It will not have XtremIO's data services though, such as thin provisioning, deduplication, compression and snapshots.
EMC II president and evangelist Chad Sakac blogged:
"There was a THIRD critical thing we learnt through Project Thunder. In many cases, the ideal interface isn’t RDMA over IB/Ethernet (memory mapping).
Rather, the way the developer DESIRED to interact with the persistence layer was directly with the application API. You could do this with NVMe, and it could be possible to 'bolt right in' to HDFS, key value stores and others. This is what DSSD does."
The DSSD technology is true “top of rack, pooled server-memory-flash”, and will provide millions of IOPS.
The DSSD technology involves many chunks of flash, each with its own controller, and with memory (RAM) to buffer or cache the flash. Hosts (app-running servers) access the DSSD flash stores through "a super-low-latency interconnect."
That implies to El Reg something different from InfiniBand or 100GbitE. Are we looking at Intel's Silicon Photonics, the Omni-Path architecture?
We're looking at much faster access to data than the networked XtremIO array can provide.
DSSD software will manage and pool this flash and automate aspects of its operation. Sakac said both block and file access stacks could be layered on top. Desai's comment confirms that block access has been added and that would widen the use cases for the tech. Imagine any enterprise app currently accessing lots of data in a SAN and needing very much faster access – step forward DSSD.