Double-DIMMed XPoint wastes sockets
It makes me cross
Analysis A Xitore white paper compares coming XPoint DIMMs and Xitore's own flash DIMMs, and claims each XPoint DIMM needs a companion DRAM cache DIMM, obviously halving XPoint DIMM density.
The startup has its own tech to push – NVDIMM-X – but, even so, is revealing about XPoint DIMMery.
Doug Finke, director of product marketing at Xitore, claims that an XPoint DIMM, expected in 2017/18, will have a small DRAM cache located on a separate DIMM module. Finke explains the reason for having this DRAM cache*:
The 3D Xpoint DIMM uses an external separate standard DRAM DIMM module as a write cache architecture and is required because the write accesses are much slower than the read accesses in 3D Xpoint media.
Here is a Xitore graphic showing its point.
We understand from Micron's Nicolas Maigne, EMEA regional business development and marketing, that any Micron QuantX DIMM would be similarly encumbered.
Intel documentation says:
Future 3D XPoint DIMMs may make it practical for main memory to hold terabytes – 6TB (6,000GB) is predicted. 3D XPoint DIMMs will probably have a slower bandwidth than double data rate (DDR) DIMMs, perhaps with their contents cached in MCDRAM (multi-channel DRAM), HBM memory to compensate for this. Such DDR DIMM caches could be about 10 per cent of the capacity of the main memory, so these caches can be 600GB in size – a far cry from the 4KB main memory on the machines from the early 1970s.
If this pairing of XPoint DIMM and a DRAM cache DIMM is correct then several consequences follow:
- For every XPoint DIMM two DIMM slots are needed, effectively halving the potential XPoint DIMM capacity on a host.
- Memory bus capacity is needed to transfer data from XPoint DIMM to cache DIMM.
- XPoint is a backing store to a cache DIMM and effective caching algorithms can make alternative and less expensive backing stores more attractive.
This third point seems to us storage peeps at El Reg to be crucial. Assume a 5 per cent cache miss rate with 1 million IOs using this scheme. Then 50,000 IOs will happen at the speed of the backing store and 950,000 will happen at (cache) DRAM speed. Let's further assume DRAM access speed equals 1 time unit and XPoint access speed equals 5 time units. Then the total access time can be calculated as:
((950,000 x 1) = 950,000) + ((50,00 x 5) = 2,500) = 1,200,000 time units.
The average time per access is 1.2 time units.
Let's employ flash DIMMs instead of XPoint ones, with an access time of 50 time units, 10 times slower, and use the same DRAM caching scheme and hit rate. What is the total access time for 1 million IOs?
((950,000 x 1) = 950,000) + ((50,00 x 50) = 25,000) = 3,450,000 time units.
The average time per access is 3.45 time units. The difference from 1.2 is significant, being almost three times longer. And this leads to a question: how much extra will you pay for XPoint DIMM speed?
Finke has this to say about XPoint cost: "It is touted to have a cost of about one-half that of DRAM, but still 5x that of NAND." Will you pay five times as much for a near 3X speed boost? We imagine that any accompanying DRAM cache DIMM would cost extra, effectively putting up the XPoint DIMM cost. So you might have to pay more than 5X the NAND price."
"Obviously this is a crude calculation with many assumptions but a message is clear: DRAM caching won't hide the latency difference between XPoint and other non-volatile DIMM technology speeds."
"We asked Intel what it thought about this use of a DRAM cache DIMM accompanying XPoint DIMMs and a spokesperson said: "Intel does not comment on unannounced products or rumors and speculation. We’ll have news regarding 3D XPoint-based products in 2017 – stay tuned." We will." ®
* Refer to this Xitore white paper "Comparison of the NVDIMM-X with 3D Xpoint in a DIMM Form Factor" authored by Finke. It discusses more than just the latency issue we have highlighted.