Erasure coding startup springs forth from Silicon Fjord
Viking raider's longboat filled with erasure codes
Analysis Memoscale is a 6-person Norwegian startup, based in Trondheim, that has developed its own erasure coding (EC) technology. It says it's more efficient than classic erasure coding because it needs fewer hardware resources to run and enables higher storage capacity utilization.
All six staff members founded the company in 2015, but actual project work started in 2013 with error coding optimization for networking. Data storage was subsequently seen to be a better opportunity.
The CEO is Per Simonsen, previously the CEO at UniTimer AS. Katina Kralevska is a PhD student at Telematics-NTNU (Norwegian University of Science and Technology), which university was also attended by Simonsen. Kralevska's PhD thesis was entitled "Applied Erasure Coding in Networks and Distributed Storage" and submitted in August 2016. Software developer Kjetil Babington is also an NTNU graduate and spent time at UniTimer.
MemoScale's chairman is Eric Wold who is Head of Venture at NTNU Technology Transfer AS, and NTNU Technology Transfer AS is MemoScale's largest shareholder. NTNU professor Danilo Gligoroski is on MemoScale's board.
MemoScale is a company based on IP generated at NTNU by NTNU students, and it is being helped to become a viable business by NTNU technology-to-business transfer funding and expertise.
It claims that normal or classic EC (Reed-Solomon) has a 1.5x storage capacity overhead, whereas its technology only has a 1.2x overhead. Classic EC method facilitates a 13MB/sec recovery speed but Memoscale IP can provide a 34MB/sec speed.
Memoscale erasure coding characteristics (Source: Philippe Nicolas)
Simplistically playing about with numbers we could say that, with a 500 drive array, classic EC with its 1.5x overhead leaves the capacity of 333 drives available for raw data, whereas Memoscale's own EC leaves 416.6 drives available – an approximate 83 drive advantage.
The recovery performance advantage in large arrays is significant, as disk failures and subsequent recoveries are a way of life in them. The combination of better storage efficiency and faster recovery should be appealing, as inefficient RAID protection gives way to EC and users/suppliers search for the least overhead and shortest recovery time commensurate with the protection levels they need.
Memoscale has four products:
- Erasure coding C-library
- Erasure coding plugin for Ceph
- Erasure coding plugin for HDFS
- Erasure coding plugin for SWIFT
The C-library product is said to be highly optimized for Intel and ARM processors, and Memoscale says it features:
- Unmatched encode and decode speed
- Support for multiple types of erasure codes
- 3X faster recovery of drives and nodes using proprietary codes with reduced recovery traffic
- Optional bit error detection and correction on reads
Memoscale has, it says, improved data scrubbing operations, as well as an interface for end-to-end checksum protection. Its software can support Reed-Solomon EC so users need not be locked in to Memoscale. It can process Reed-Solomon EC faster than other approaches such as Intel's EC library and Jerasure.
The Library and Ceph products are available now, with the HDFS and SWIFT plugins coming soon.
MemoScale has been granted around 10 million NOK from the Research Council of Norway and NTNU Discovery. More funding is being sought.
The company can sell its software to large Ceph users and is also looking to forge partnerships with storage suppliers. Kaminario has mentioned it has had discussions with Memoscale.
Simonsen says he is in the US once a month visiting potential partners. So watch out, a Viking raider is about; crossing the Atlantic again, with his metaphorical longboat filled with erasure codes and bent on replacing Reed-Solomon. ®