NetApp adds Lustre to Big Data line-up

SC11 NetApp gunning for HPC and Big Data business with two product bundles combining its E arrays with Lustre, the open source supercomputer file system.

At the same time the company has teamed up with Whamcloud, a VC-backed firm "formed from a worldwide network of HPC storage industry veterans," to promote the Open Benchmark for Lustre (OBL). They say this should help customers compare suppliers' Lustre storage offerings.

My colleague Tim Prickett Morgan has a good run-through of the Lustre File System here. So let's concentrate on today's announcements.

First up is NetApp's High Performance Computing Solution for Lustre (HPCSL). This is said "to efficiently scale bandwidth and density" with 30GB/sec writes. There are FastStak and DenseStak (pdf)reference configurations with FastStak configured to optimise E5400 storage performance with high bandwidth and IOPS. DenseStak is configured to optimise storage density for maximum capacity in a 40U rack, using 10 x 4U E5400 enclosures holding five drawers of 12 drives each for a total of 600 drives.

4U NetApp DenseStak enclosure

Next up from NetApp is a Seismic Processing Solution (SPS) bundle (pdf). This has up to 1.8PB of E-Series storage in a 40U rack, along with Quantum's StorNext file virtualisation system. It provides 4.4GB/sec bandwidth from a 4U enclosure, and the system scales by adding these enclosures. A full rack delivers 50GB/sec read and 35GB/sec write bandwidth.

A fully loaded FastStak rack delivers performance of up to 100GB/sec sustained disk read throughput, 70GB/sec sustained disk write throughput, and 1,500,000 sustained IOPS.

Benchmark this

So why is NetApp promoting benchmarks for Lustre? We can infer that NetApp thinks its gear will perform pretty well but aside from this Eric Barton, CTO at Whamcloud, argues that an industry-standard benchmark is needed to provide common configuration guidance and performance reference points."

The OBL will provide, NetApp and Whamcloud say, "a framework that enhances configuration guidelines for optimised installations ... the benchmark will help to frame ... deployment decisions for improved efficiencies in areas such as bandwidth, efficiency, power consumption and floor space."

The initiative is conceived as an industry collaboration, encouraging vendors to identify benchmark configurations that are transparent. The benchmark data is then published for apples-to-apples€ comparisons of Lustre in different storage environments.

To join the working group and or to find out more contact: info@openbechmarkforlustre.org. ®