Up the stack with you: Microsoft's Denali project flashes skinny SSD controllers
The goal? Commoditise the data centre NVMe SSD world
Analysis Microsoft has lifted the lid on Project Denali, its quest to bring flash drive cost down a peg by extracting costly extra software from SSDs and running it in the host server to gain cost and performance efficiencies.
It and co-developer CNEX Labs presented details about the project, along with its SONIC open switch networking concept, at last week's Open Compute Summit, an event dedicated to making IT gear cheaper for hyperscale buyers like Facebook, Amazon, Google and Microsoft itself.
Denali is a reference architecture for NVMe SSDs which removes software functionality found in many SSDs today and sends them up the stack. The functions include address mapping, garbage collection and wear-levelling.
The SSD controller carries out more basic tasks like bad block and media management and power failure.
With the upper-level functions running in a host server, Microsoft says the adoption of coming solid-state storage technologies, such as QLC (4 bits/cell) flash and SCM (storage-class memory), will be faster. Only one piece of code will have to be modified to support them instead of each drive supplier developing their own code to carry out the job. It will be like NVMe code replacing individual suppliers' PCIe drive software.
Microsoft has stated: "Project Denali will allow hardware companies to build simpler, less complicated hardware which will lower costs, decrease time to market, allow for workload specific tuning and enable rapid development of new NAND and memory technologies."
There are two ways the Denali project envisages the displaced upper-level SSD code executing. One is for the code to run directly on a host server. The other has it executing on an FPGA or SoC (System-on-Chip) device. The FPGA/SoC would be a host CPU offload accelerator device.
A Microsoft diagram shows these options:
Microsoft's Kushagra Vaid, general manager, Azure Hardware Infrastructure, wrote of Denali:
... customers can achieve greater levels of performance, while leveraging the cost-reduction economics that come at cloud scale.
Project Denali will, if adopted, tend to commoditise data centre NVMe SSDs. According to Wells Fargo senior analyst Aaron Rakers, who was at the summit, the major cloud suppliers consume between 15 and 20 per cent of total worldwide flash bits production.
He also said that Denali may validate the approach of all-flash array vendors, such as Pure Storage, who already locate their garbage collection, wear-levelling, etc. in an array controller and not in the component SSDs. They claim to get better overall flash drive performance efficiency by doing so.
The Denali project was demonstrated at the OCP event, and will be offered to OCP as a standard when it is finished.
How will the SSD makers react to this?
Microsoft has several Denali partners, including SSD suppliers Intel, Samsung and SK Hynix, and SSD controller maker Marvell.
Projects like Denali are submitted to the OCP Incubation Committee and OCP technology area project leads help drive the process.
Western Digital's Jorge Campello, senior director of Systems and Software Technologies, is the OCP Storage project lead.
Seagate is a gold-level OCP member and Toshiba is a community-level member. That means, firstly, all the main SSD suppliers are formally supportive of the OCP. Secondly, Intel, Samsung and SK Hynix are already on-board with Denali. Thirdly, Western Digital will be in a position to help usher Denali to the OCP's incubation committee.
The SSD makers would look to increased SSD sales to hyperscale buyers as a way of making up for any lost income from forfeited added-value SSD technology, like on-drive garbage collection and wear-levelling. ®