Flashy fists fly as OCZ and DDRdrive row over SSD performance

CTO says rival's specs are 'knowingly disingenuous'

  • alert
  • submit to reddit

HP ProLiant Gen8: Integrated lifecycle automation

Two solid state disk SSD suppliers are arguing about NAND flash performance drop-off.

OCZ supplies NAND flash solid state drives (SSDs) and regularly announces high-performance products. DDRdrive has recently exited stealth mode and makes the X1 hybrid DRAM/NAND SSD. It criticises OCZ and other flash suppliers for products exhibiting a dreadful performance decline after a little use fresh out of the box.

Christopher George, founder and chief technology officer of DDRdrive, talks of "dirty tricks used to drastically inflate IOPS results", "the egregious disparity between promised performance and reality", and "the untold truth about OCZ's *sustained* write IOPS".

George gave a presentation at the Open Storage Summit 2010 on flash SSD performance degradation and we have received a copy of the slide deck. The PowerPoint pitch asserts that "Flash SSDs produce one-time, unsustainable, dramatically inflated results when tested 'new' or after a Secure Erase."

Then George wades in on OCZ, claiming: "Both the OCZ Vertex 2 EX and the OCZ Vertex Pro SSD use the SandForce 1500 controller, which implements compression at the drive level. Benchmarking with an Iometer version that defaults to using extremely compressible data will, in these cases, show dramatically inflated write IOPS results."

He used pseudo random I/O transfer data in Iometer v 2006.07.27 instead, which, he says, provides generalisable results, and tested the X1 and OCZ's Vertex 2 Ex and Pro devices. A slide showing IOPS performance sequentially writing 4KB blocks over time shows a pattern that is repeated again and again with other I/O activities, such as random writes.

DDRdrive vs OCZ Vertex EX 2

4KB Sequential writes after Secure Erase and over one hour's sustained testing.

The Vertex 2 EX initially writes at 50,000 IOPS but after six minutes or so the IOPS number falls off a cliff and drops to 13,000 - 15,000 IOPS where it stays for test after test from seven minutes to an hour after the test session started, and continues at a 12,000 - 10,000 IOPS level from 120 minutes to 180 minutes after the test started.

George sums up his findings like this: "Flash based SSDs succumb to dramatic write IOPS degradation in as short as 10 minutes after [the] device is unpackaged or Secure Erased."

OCZ: 'Not an apples to apples comparison'

OCZ says the Vertex EX 2 "uses Single-Level Cell (SLC) memory, which has 10 times the program/erase (P/E) cycles than MLC-based drives, and features superb 4k random writes up to 50,000 IOPS to give a performance edge over the previous generation."

The supplier says: "Rated speeds may vary slightly depending on the benchmark used, drivers, windows version, bios version and file size. We recommend using ATTO, IOMeter, and PC Mark Vantage for benchmarking SSDs and to achieve maximum rated specifications."

OCZ spokeswoman Jessica Luken said: "This really isn’t an 'apples to apples' comparison, and I know exactly why this company would compare its solution to SSDs. My boss put it into perspective with this analogy: 'It is like comparing a drag car designed to accelerate fast in straight lines to a Ferrari which may not be the fastest vehicle but is built to negotiate the track'."

Luken said: "It is no secret that one of the attributes of NAND-based storage devices is a change in performance from new, fresh-out-of-box (FOB) state to 'pre-conditioned' state. All NAND-based drives would exhibit similar results in this comparison ... The OCZ Vertex2 EX series of drives is among the best in the industry at maintaining close to FOB performance," she said.

"There are, of course, corner cases that show a larger drop in performance. These corner cases – when paired with a platform and software environment that is not NAND SSD optimised – can be made to appear more significant than real world scenarios. OCZ's FAE [Field Application Engineering] team works closely to enterprise customers to ensure their environment is optimised to maintain best-in-class performance and endurance."

She added: "I’d also argue that the benchmarks provided are not accurate, as the testing doesn’t even exhibit what all other third party media are reporting, and it’s clear the testing was definitely run on an unTRIMed drive in dirty state."

OCZ Field Applications engineer Mark Hayashida added: "Though it is true that performance of Flash-based drives can change from its 'new' condition (or after a Secure Erase) over time due to certain conditions, this [DDRdrive] presentation is only looking at a particular area that calls out some worst case scenarios for Flash-based drives, such as ZFS and its intent log (ZIL) performance...

"Typically (what I call an I/O Signature of) ZIL is usually spiky in nature, not always sustained, so with TRIM support, when it comes to ZFS, that should help performance on Flash SSDs running ZIL. That would provide time for the drive to do some 'housekeeping' and maintain a higher level of performance over the life of the drive."

'Specifications are knowingly disingenuous'

Regarding TRIM, a means for Windows to tell the SSD controller about deleted data in the SSD, DDRdrive's George said: "I did run the exact same tests on Windows 7 (which does support TRIM) and the OCZ results do marginally increase. But the overall trend line stays constant; *dramatic* IOPS degradation relative to published specifications. Basically, unless one uses a development version of Iometer, which happens to default to a contrived data pattern, the published specifications are knowingly disingenuous."

The OCZ pitch is that we should use the right SSD for the right job. The X1 may well be great at ZIL performance and the Vertex 2 EX less suitable, but that does not mean you can generalise from ZIL to everyday flash SSD application performance. In other words, you can't use an X1 dragster on the open road but you can use a Ferrari Vertex 2 EX. ®

Reducing security risks from open source software

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
prev story


Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.