Dell EMC's Pure-crushing benchmarks are flawed, says, er, Pure

Lies, damn lies, and performance test results

We've seen selected benchmarks that show Dell EMC Unity arrays giving Pure a hard time – and, unsurprisingly, Pure says the tests are "obviously flawed."

The Unity arrays are reengineered VNX arrays that were introduced in May. All-flash versions were used in the two benchmarks below. An obvious caveat is that these are selective benchmark results and we don't know if we have seen the whole set of results. What we have seen indicates that the Unity array can outperform Pure Storage Flash Arrays.

The benchmarking software followed, we're told, IDC and SNIA testing recommendations for all-flash arrays. It involved varying percentages of reads and writes, different block sizes, and 100 per cent random workloads. Dell EMC believes this represents a typical real-world use case, whereas comparing millions of 4K 100 per cent read sequential IOPS does not.

Here are the results we have seen:

Workload Pure //m50 Unity 600F Unity Advantage
16K IOPS (80% Read) 33,460 58,807 1.8X
256K BW MBps (@3ms) 674 2,396 3.6X
Steady-state IOPS 42,000 116,000 2.8X
Steady-state latency (ms) 13.6 4.4 3X

The tested configurations were:

  • Unity 600F
    • 2 active-active controllers
    • 4 x 16Gbit/s FC ports
    • RAID5 (8 + 1)
    • 16 LUNS, 32 or 16 threads
    • 19 SSDs
  • FlashArray //m50
    • 2 controller (active-passive backend)
    • 4 x 16Gbit/s FC ports
    • RAID-3D
    • 16 LUNs, 32 or 16 threads/LUN
    • 40 SSDs (estimated)

We saw a Unity vs FlashArray //m70 benchmark result as well:

Workload Pure //m70 Unity 600F Unity Advantage
8K IOPS (80% Read) 31,448 163,741 5X
256K BW MBps 540 (@c5ms) 3,450 (@c5ms) 7X
Steady-state IOPS 12,000 86,000 7X
Steady-state latency (ms) 64 5.9 11X

Here are the tested configurations:

  • Unity 600F
    • 2 active-active controllers
    • 12 x 16Gbit/s FC ports
    • RAID-6 (10+2, 10+2, 10+2, 6+2)
    • 64 LUNS, 8 threads/LUN
    • 47 SSDs (44 + 3 spares)
  • FlashArray //m70
    • 2 controllers (active-passive backend)
    • 8 x 16Gbit/s FC ports
    • RAID-3D
    • 16 threads/LUN
    • 40 SSDs (estimated)

Pure's Matt Kixmoeller, VP of products, said: "Pure always encourages customers to perform real [proof-of-concept tests] with their application data, as synthetic testing has proven [over] time to be a poor indicator of the comparative performance of any AFA. In this case, the alleged test results are so obviously flawed that they serve to discredit themselves. For example, the tests seem to indicate that Pure's //m50 delivers higher bandwidth than the higher-end //m70 (which is an impossibility)."

Kixmoeller continued, "When doing any head-to-head testing, it is important to understand a myriad of configuration questions, such as whether or not compression and deduplication were enabled, whether encryption was enabled, how full the array was, whether data services like snapshots or replication were enabled, and whether the test data set has similar data reduction and locality characteristics to the actual planned workload.

"Pure's products are designed to deliver the highest levels of consistent performance, even with all data services engaged. We encourage customers to perform their own objective [proof-of-concept] comparisons, ideally with their own application data, and we can further enable customers to implement best practices for an all-flash environment, rather than bring along disk environment baggage." ®

Sponsored: The Joy and Pain of Buying IT - Have Your Say


Biting the hand that feeds IT © 1998–2017