Quantum takes on GPFS and Lustre in commercial HPC market
Colliding parallel file access products
Analysis The entertainment and media market has not been typically seen as a part of the high-performance computing (HPC) market, with the associated massively parallel file access and data set management. In the HPC world file access software such as Lustre and GPFS, now renamed Spectrum Scale are often seen.
HPC was an academic and supercomputing speciality but has spread into commercial enterprise use as businesses in the oil and gas and life sciences areas turn to HPC to crunch through the ever-growing volumes of data they meed to process. There we see lots of workers running tasks on fresh data in parallel, with older data archived off to back-end stores and workflow structures set up to manage the data and processes.
The entertainment and media world has moved in the same direction with special effects. As film resolutions have increased so has the reality of special effects, with digital artists creating the vividly detailed and multi-layered backdrops for human actors, and also virtual actors. This is the world in which Quantum’s scale-out StorNext file access management and virtualisation product set lives.
The complex tiered world inside StorNext
It provides primary file data storage and access with data ingest, and older files stored on nearline (object storage) disk or tape or the public cloud, and files being moved by workflow processes through these storage tiers as needed, both to keep the digital artists fed with data and the storage repository costs down by using less expensive storage tiers for data that doesn’t need immediate access.
Quantum says it has found StorNext use spreading to video surveillance, the Transportation of Things (eg, autonomous vehicles) and life sciences and other vertical markets where HPC is finding a role.
StorNext does not compete in the traditional HPC market, working in small to medium clusters in the HPC scale. But, according to Quantum’s marketing VP Molly Rector, it does compete with Spectrum Scale and Lustre in the commercial HPC (HPC-lite?) area.
NAS systems can run into difficulties when data sets grow past a petabyte in size. Here the scale-out StorNexts, Lustres and Spectrum Scales contain NAS (filer) sprawl in data-intensive environments.
Rector joined Quantum in September last year, when the company was making progress growing its StorNext business as growth faltered in its DXi data protection products and continued being negative in tape products.
For StorNext to continue to grow, it has to spread outwards from its entertainment and media-type core and go into wider enterprise use. So it is positioned as a NAS sprawl rescue product and has to be chosen by customers instead of alternatives such as SpectrumScale and Lustre. Quantum has recruited HPC-type expertise with, for example, sales specialist Jason Coari joining from Adaptive, with SI employment before that.
The world in which StorNext has to operate is like an equivalent of a complex manufacturing process: data manufacturing with carefully defined and managed, layered workflows of digital components being processed and fitted together to form a finished digital product, a film or video, an oil field model, or a genomic analysis. The design and costing and operation of the infrastructure behind such manufacturing process flows is crucial, and Quantum is hoping that its StorNext experience in the entertainment and media heartland will enable it to make progress in allied data-driven digital product manufacturing markets.
It’s game-on as scale-out parallel file access suppliers aim to take over where traditional filers falter and fall behind. Customers will buy systems based on performance, acquisition and operational costs, and workflow flexibility, maturity and fit.
Suppliers like DDN, Dell with Isilon (all-flash version due) and the coming XtremIO filer, Avere, and Qumulo will be butting up against SpectrumScale and Lustre suppliers. Quantum is now in this race with them to be a scale-out, digital data product manufacturing workflow supplier. ®