Three's a cloud: Toshiba picks its NVMe over Fabrics storage node
Kumoscale software presents fabric access NVMe flash drives virtually
Surprise, surprise – flash chip and SSD manufacturer Toshiba has announced NVMe fabric-access flash array software. What's its game?
An NVMe over Fabrics (NVMe-oF) array has a bunch of NVMe flash drives inside it which are accessed by applications running in servers. These apps think they are accessing local, NVMe-accessed, PCIe-attached SSDs. Instead they are accessing remote NVME SSDs using a remote direct memory access protocol that should go direct to the drive for the lowest possible access latency – 50 microseconds for an E8 array write for example.
At a briefing we were told about KumoScale*, a software product implementing the target end of an NVME-oF storage network. A Toshiba diagram lays out its components:
We have a rack of servers accessing a box of flash drives with KumoScale software sitting in the data path at the front of the box. It virtualises and abstracts the SSDs, turning them, we were told, into one exceptionally large SSD.
This software is integrated via its RESTful API with data centre orchestration frameworks such as Kubernetes, OpenStack, Intel RSD and also Lenovo XClarity. There is also a Portworx angle here but we’ll look more at what KumoScale itself does before getting to that.
There is a supporting quote from Lenovo's Chris Verne, executive director at its Data Center Group: "We've recently collaborated on the KumoScale software solution, demonstrating its ability to work on Lenovo’s platforms. We see the Toshiba software solution as a key innovation for future NVMe-oF offerings that enable increased flexibility and efficiency."
A Toshiba slide deck says Kumoscale's attributes include:
- Abstracts details of physical NVMe SSDs; provisions any size volumes
- Secure ACL maintains and specifies access rights to NVMe resources
- N x 100G Ethernet network ports; full network bandwidth at 4KB random I/O
- <20 μS latency adder
- 8,000,000+ 4K Random read IOPS
- 4,000+ namespaces
- 384TB SSD capacity
- Popular cloud frameworks
- Broad ecosystem support
- 100 per cent elastic
- Zero stranded capacity
We pick up from this that Toshiba has a flash capacity limit – 384TB SSD – and we understand KumoScale runs on a Linux x86 server, any industry-standard NVMe server platform, which it turns into a shared block storage device.
Data access latency with KumoScale is not quite 20μS (4K data block) longer than if KumoScale was not in the datapath.
More performance data:
The zero stranded capacity refers to its role as a DAS replacement, where DAS SSDs can be under-used. Not so with KumoScale, claims Tosh, where the SSD capacity is pooled and shared.
Toshiba first announced this software in August last year, when we wrote: "The software abstracts the details of physical SSDs to enable them to assign virtual namespaces of any size and performance to any compute node making NVMe-oF more deployable in cloud data centres."
KumoScale is targeted at cloud-centric customers and hyperscalers: people who buy 5,000 servers at a time. We're told by Toshiba that it is suited for cases where the customer has a product or service delivered over the web and suffers from demand peaks. The customer has to buy enough machinery (servers + DAS) to handle the demand peaks and the excess kit sits idle when the peaks go away.
By using a KumoScale shared NVME flash box as a networked storage node, the customers gets the performance needed without this idle machinery overhang.
Toshiba will not ship a hardware+software product. Instead it will work with a customer's preferred server supplier and certify that supplier's hardware. It will be fulfilled by the customer's supply chain with KumoScale supported by Toshiba.
The software was written at the Toshiba Israel Storage Design Center, run by general manager Oded Ilan.
Deeply troubled SSD supplier OCZ sold itself to Toshiba in December 2013 for $35m. The OCZ assets included an R&D Centre in Tel Aviv that was concerned with micro-controllers and software. It was one result of OCZ buying Sanrad in January 2012 and getting its hands on its virtualiZed server flash cache software. This farmed out server flash to virtual machines.
OCZ bought Sanrad from RAD Data Communications. Oded Ilan, who was president of RAD North America up to July 2008, then became Sanrad CEO, made the move to OCZ, and now runs the Toshiba Israel Storage Design Center.
KumoScale has an integration with Portworx, which provides dynamically provisioned Kubernetes storage, and has its volumes run on top of KumoScale's target namespaces.
Toshiba Memory America and Portworx will jointly demonstrate KumoScale at OCP booth #A21 at the San Jose Convention Center from March 20-21.
* "Kumo" is cloud in Japanese