NetApp modifies benchmark system and - shock - comes out TOP
Maybe it shouldn't be called 'Doing a Dot-Hill' any more
NetApp's number crunchers at Sunnyvale have sweetened the firm's SPC-2 benchmark results in an attempt to help NetApp stand out from the crowd, emphasising the price/performance bennies of its latest E5500 disk array.
The Storage Performance Council (SPC) is a storage industry benchmark body that helps the industry provide standard and objective storage array transaction performance (SPC-1) and large-scale, sequential data movement performance (SPC-2) test results. With SPC-2 data suppliers can say their system moved so many MB/sec of data with a particular price/performance level. These benchmark results enable valid and accurate comparison's to be made between different suppliers' storage arrays when performing the benchmark task.
What NetApp's numerical alchemists have done is to refine the good SPC-2 result they obtained with their E5500 storage array and amplify the difference between it and other systems by using a new measure calculated from the benchmark data. Insiders call it "doing a Dot-Hill" after that supplier employed similar creativity to make its array stand out from the crowd.
The E5500 is NetApp's latest Engenio E-Series array, and aimed at the big data and high-performance computing (HPC) market. SGI is OEM'ing the E5500 disk array controller and populating it with drives, calling the result its IS5600. The E5500 controller has an ASIC instead of the expected X86 commodity processor controller. It can manage 384 drives compared to the previous model's 192 drives and scale to 1.2PB instead of 576TB, delivering on the whole about 2.5X more IO performance.
NetApp product and solutions marketing veep Brendon Howe said NetApp had set out to "build a new product that provides industry-leading bandwidth per dollar spent while improving density and reliability." The Sunnyvale headquarted company said SGI had run an SPC-2 test of its IS5600 which "demonstrates the highest throughput per spindle by more than 2.5 times over the nearest non-NetApp published result."
There is no such thoughput-per-spindle SPC-2 measure, the council's test results instead providing throughput in MB/sec (MBPS) and price/performance; the configuration's list price divided by the MBPS figure. The chart below shows a set of SPC-2 results which include the SGI IS5600.
SPC-2 benchmark results. Lower right corner is price/performance goodness. Click on chart to get a larger view.
The vertical axis is price/performance in dollars; the lower the better. The horizontal axis is MBPS; the higher the number the better. NetApp has the third best MBPS number and the absolute best price/performance from a set of SPC-2 results. Here's the price/performance rankings:-
- $15.97 SGI 5600 NetApp E5500
- $28.48 Fujitsu ETERNUS DX80 S2
- $35.24 Sun ZFS Storage 7420
- $49.37 TMS RamSan-630
- $66.50 Fujitsu Eternus DX440 S2
- $71.32 IBM Storwize V7000
- $152.34 IBM XIV
- $270.38 IBM DS8800
This wasn't a good enough way of showing the E5500's data-moving prowess, it seems - so NetApp's number-crunchers devised a new way of presenting the results, dividing the MBPS total by the number of disk drive spindles in the tested configuration, and so came out on top of the list by a wide margin.
NetApp's MBPS/spindle SPC-2 table
Now NetApp can say the test results "demonstrates the highest throughput per spindle by more than 2.5 times over the nearest non-NetApp published result."
Well, not quite. We added in the XIV result, using data from an SPC-2/E energy efficiency version of the benchmark. NetApp is still ahead by a good margin though, at 73.80 MBPS per spindle.
We also added the all-flash TMS RamSan 630 to the table to indicate that a MBPS/spindle measure is useless for an all-flash array.
The NetApp statisticians' wizardry shows that the E5500 ASIC and firmware can move sequential data on and off disk drives faster than any other storage array in this particular set of tested systems.
There you have it; NetApp's nabobs of numerology have spun the SPC-2 stats and lifted NetApp above the also-rans at the top of the SPC-2 sequential data moving heap.
Whether you're buying it is up you. ®