NetApp: Steenkin' benchmarks – we're quicker than 3PAR

Playing the latency card, NetApp says it has a faster array than 3PAR – despite having a lower SPC-1 benchmark score.

A 6-node FAS6240 cluster SPC-1 benchmark (PDF) result has been posted. SPC-1 is a storage array benchmark measuring the number of IOs per second (IOPS) to data accessed over Fibre Channel as blocks rather than files.

Chart of SPC-1 benchmark results

The NetApp benchmark in itself is midrange in terms of its score of 250,039.67 IOPS, as the chart shows. But an IBM Storwize V7000 has achieved 520,043.99 IOPS. NetApp supplies a full list price for its system, $1,672,602, giving a $6.69/IOPS price/performance rating. It says other suppliers discount by up to 50 per cent compared to their supplied prices. Doing the same gives us $3.35/IOPS, which is better than average for this IOPS performance as the $/IOPS chart below shows.

SPC-1 $/IOPS chart

There are eight systems faster than this FAS6240 cluster in the SPC-1 listings, but NetApp argues that at the 3ms latency level its cluster is faster than a higher-ranked HP 3PAR P10000 array, which scores 450,212.66 IOPS. Why? NetApp says: "These are among the best SPC-1 results for enterprise disk-based systems given the low latency for the IOPS provided."

The SPC-1 benchmark requires latency numbers to be provided at various percentage workloads and NetApp has tabulated its own IOPS along with some competitors' IOPS at the 3ms latency level.

NetApp's SPC-1 latency table at c3ms

The table shows the 6-node FAS62490 cluster scoring 250,039.97 IOPS with a 3.35ms latency and the 3PAR P10000 doing 225,079.30 IOPS at 3.29ms – which would make NetApp's array faster than 3PAR's at this latency level. NetApp admits that an IBM SVC configuration did better than its cluster – 259,998.37 IOPS at 2.71ms – but says:

The IBM result is a great example of a very fast system built by adding a lot of hardware together:

• 8 SVC virtualisation engines, plus
• 16 separate V7000 systems under the SVC controllers...
• each consisting of 2 more SVC controllers and 2 RAID controllers
• plus 1,920 146GB 15,000 RPM disks (not quite the drive type people buy these days)
• [All] for a grand total of 40 Linux-based SVC controllers (8 larger and 32 smaller), 32 RAID controllers, and a whole lot of disks.

  Compare it to the NetApp configuration:

• 6 controllers in one cluster; and
• 432 450GB 15,000 RPM disks (a pretty standard and common drive type as of the time of this writing in June 2012).

In a nutshell it's very much more complex and very much more expensive.

3PAR P10000 latency curve on SPC-1 benchmark

NetApp also says its systems degrade more gracefully as the workload increases. It shows two latency charts: one for its system (below) and one for the 3PAR P10000 (above). Indeed, the 3PAR curve steepens significantly as the workload increases to the 100 per cent SPC-1 maximum.

NetApp 6-node FAS6240 cluster SPC-1 latency curve

However, El Reg would like to point out the results for a TMS RamSan-620 (PDF) solid state array from December 2009. It did 254,994.21 IOPS at a cost of $1.13/IOPS and its latency chart is both lower and flatter than NetApp's as the chart below shows.

TMS RamSan-620 SPC-1 latency curve

A newer RamSan-630 scored 400,503.26 IOPS at $1.05/IOPS. It costs $419,212 compared to the 6-node FAS 6240's $1,672,602 list price, $836,301 at a 50 per cent discount.

Of course, with the RamSan you don't get all that NetApp data management goodness, or the ability to have your array operate as an iSCSI box and a filer box simultaneously. NetApp has demonstrated that its ONTAP 8 FAS array clusters perform respectably, to say the least, at block data access, and that it has nice and flattish latencies as the workload increases compared to its picked out competitor 3PAR. Part of the reason for this is the 512GB of FlashCache incorporated in each FAS 6240 controller.

If we look at all-flash systems, like the RamSans, we see high performance, the best latency numbers in the SPC-1 tables and the best $/IOPS numbers too. However well you manage your disk spindles – and NetApp manages them very well indeed – and however many you have, the flash array suppliers are closing the absolute performance gap, delivering both lower latencies and better SPC-1 price/performance numbers. ®