Upstart chip chef Diablo's DIMM yum dumplings cram NAND in RAM

Solid-state storage wired into DDR3 bus

Internet Security Threat Report 2014

Canadian storage startup Diablo has pulled out what it hopes will be a PCIe-flash killer: its new tech provides fast flash data access by interfacing a NAND partner's solid-state drives to a host server's memory bus.

Diablo, founded in 2003, has invented what it calls Memory Channel Storage (MCS) in which NAND flash is packaged in DIMM (Direct Inline Memory Modules) units called TeraDIMMs, and slotted into a server's DIMM sockets. A partner provides the NAND flash and will be the sales channel for the TeraDIMM (or whatever the ultimate brand name is).

The TeraDIMMs have ultra low latency compared to SSDS and PCIe flash cards.

Described by Daiblo's patent application:

a TeraDIMM is a new class of DIMM memory that enables the creation of physical memory systems that can match the virtual memory capabilities of CPUs by integrating non-volatile memory (eg, NAND flash, phase-change, etc) into, for instance, a DDR-3 or DDR-4 memory channel.

Diablo was given $28m by investors in November last year to help develop its MCS chipset and its ASIC hardware and firmware technology and produce TeraDIMMs.

This uses the DDR3 protocol and the flash is addressed as part of the existing memory address space. Diablo provides the DDR3 to SSD controller and firmware but needs a flash product vendor to provide the NAND, like SMART.

We already know Diablo and SMART Storage are working together with an exclusive relationship.

In April, John Scaramuzzo, president of SMART Storage Systems, said: "The convergence of Diablo’s groundbreaking MCS technology and our award-winning Guardian Technology Platform is not just about re-architecting storage, but re-architecting the integration of servers, storage and Flash acceleration. We are setting a new standard for ULL (ultra low latency) storage in the enterprise, improving performance over currently available SSDs, including PCIe-based drives.”


TeraDIMMs fitting into a server's DIMM slots

By slotting MCS access into a partner's flash chips, Diablo claims it has made data access much faster than with PCIe-connected flash. The firm says its TeraDIMMs, which have 200GB or 400GB of 19nm MLC capacity in a chart we saw, have latencies approaching that of DRAM - let's say 5-10µs write latency - and can do 150,000 read IOPS and 65,000 write IOPS. The things can run at 10 full drive writes a day for five years, the firm tells us.

IO performance scales as more TeraDIMMs are added, but the latency stays constant. Diablo claims the latency is half that of existing PCIe flash products and provides a chart to show this.

TERADIMM Latency vs PCIe flash

TeraDIMM latency (red) versus PCIe flash latency (blue) in high-frequency trading scenario.

The case being made is that, in memory subsystem terms, PCIe flash is quite distant. With TERADIMMs data is in the memory subsystem and so access to it is faster.

The TeraDIMMs can be used for in-memory compute with an app's entire dataset in the combination of DRAM and flash. Diablo compares a 25,000 144GB DRAM servers using MEMCACHED with a 5,000 servers kitted out with 16GB of DRAM and 800GB of TeraDIMM flash, saying the TeraDIMM servers provide RAM disk performance and $31m/year in total cost of ownership savings.

Servers can have, say, 16 to 24 DIMM slots. Fill 16 of those with 400GB TeraDIMMs and we have 6.4TB of flash. What we have here is a way of providing terabytes of NAND to servers in a way that has so far been done with masses of PCIe flash such as Fusion-io's ioDrive 2 with up to 10TB using 8 x 1.2TB modules, and its ioScale with 3.2TB per card. LSI's WarpDrive goes up to 3.2TB and Micron is promising a 16TB P520 PCIe flash card in the future.

Performance comparison with PCIe flash

Jim Handy, Director at Objective Analysis, said: "Flash SSDs boost system performance, and flash over PCIe is even faster, but the best performance will come from flash on the memory channel." Diablo says its MCS architecture reduces latencies by more than 85 per cent over PCI-Express based SSDs and 96 per cent over SATA/SAS based SSDs.

Performance comparisons are tricky as a generic PCIe flash card does not generally equal a 200/400GB TeraDIMM.

Let's take a 1.2TB Micron P420m MLC PCIe flash card, which runs at 750,000 random read IOPS. A server fitted with 3 x 400GB TeraDIMMs, the same NAND capacity, would receive 3 x 150,000 IOPS — 450,000 IOPS in total — from the flash, with a read latency of, say, 5μs versus the P420m's ≤75μs.

With equivalent NAND capacity levels, the TeraDIMM reacts faster than the P420m but doesn't deliver more IOPS. That might be because the IOPS are not being counted in a consistent fashion with, for example, 4K data blocks being used in both cases.

This is just The Reg storage desk's back-of-the-envelope guesstimating. We'd need a real benchmark to see what the deliverable performance is in each case.

Diablo claims that its technology provides radically lower $/IOPS numbers than PCIe flash.

How about application software? What does it need to do to make use of TeraDIMMs?

Diablo CEO Riccardo Badalone says app software doesn't have to change at all. The Diablo-accessed NAND can be used as a cache by caching software in which case the application doesn't see it directly, or it can be a block storage device that happens to be in the host's memory address space. A driver can be hooked into it to turn it into a swap engine.

Software environments that can benefit from TeraDIMMs are said to include:

  • Database and the Cloud
  • Big Data Analytics
  • High Frequency Trading and other financial applications
  • Server and Storage Virtualization and Consolidation
  • Virtual Desktop Infrastructure

SanDisk acquiring SMART Storage

Diablo and SMART have been working on this technology for two years and it appears that SanDisk's purchase of SMART Storage came as a happy surprise to Diablo. Badalone said: "It's definitely a very positive event for us."

Happily SMART is a user of Toshiba NAND, and Toshiba and SanDisk are partners in the NAND foundry business.

The acquisition gives SMART surety of NAND supply and adds SanDisk's distribution channels to SMART's existing channels.

SanDisk has its own PCIe flash product line: the Lightning brand of 200 and 400GB SLC flash products with a gen 2.0 PCIe interface. They clearly need refreshing to PCIe gen 3.0 and, probably, to be given NVMe support. It will be fascinating to see how SanDisk positions its PCIe products with what we presume is the fruit of the exclusive Diablo/SMART Storage relationship.

Badalone says there are lots of proof-of-concept trials taking place with tier 1 end users.

At this point Fusion-io, Virident and Diablo/SMART (presumably) have flash architectures for servers that can provide multi-terabyte flash capacities for applications and enable in-memory computing. Customers are encouraged to use Fusion-io software to get maximum use from Fusion's flash hardware.

Diablo's way of providing flash-enhanced performance with no app software changes may very well appeal as an instant fix for servers in performance-intensive environments that are suffering from data access latency limitations, even where the data is coming from SATA and/or SAS-connected SSDs, or even PCIe flash cards. ®

Beginner's guide to SSL certificates

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
'Urika': Cray unveils new 1,500-core big data crunching monster
6TB of DRAM, 38TB of SSD flash and 120TB of disk storage
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
SDI wars: WTF is software defined infrastructure?
This time we play for ALL the marbles
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
Oracle hires former SAP exec for cloudy push
'We know Larry said cloud was gibberish, and insane, and idiotic, but...'
Symantec backs out of Backup Exec: Plans to can appliance in Jan
Will still provide support to existing customers
prev story


Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.