Flashy fists fly as OCZ and DDRdrive row over SSD performance

CTO says rival's specs are 'knowingly disingenuous'

  • alert
  • submit to reddit

High performance access to file storage

Two solid state disk SSD suppliers are arguing about NAND flash performance drop-off.

OCZ supplies NAND flash solid state drives (SSDs) and regularly announces high-performance products. DDRdrive has recently exited stealth mode and makes the X1 hybrid DRAM/NAND SSD. It criticises OCZ and other flash suppliers for products exhibiting a dreadful performance decline after a little use fresh out of the box.

Christopher George, founder and chief technology officer of DDRdrive, talks of "dirty tricks used to drastically inflate IOPS results", "the egregious disparity between promised performance and reality", and "the untold truth about OCZ's *sustained* write IOPS".

George gave a presentation at the Open Storage Summit 2010 on flash SSD performance degradation and we have received a copy of the slide deck. The PowerPoint pitch asserts that "Flash SSDs produce one-time, unsustainable, dramatically inflated results when tested 'new' or after a Secure Erase."

Then George wades in on OCZ, claiming: "Both the OCZ Vertex 2 EX and the OCZ Vertex Pro SSD use the SandForce 1500 controller, which implements compression at the drive level. Benchmarking with an Iometer version that defaults to using extremely compressible data will, in these cases, show dramatically inflated write IOPS results."

He used pseudo random I/O transfer data in Iometer v 2006.07.27 instead, which, he says, provides generalisable results, and tested the X1 and OCZ's Vertex 2 Ex and Pro devices. A slide showing IOPS performance sequentially writing 4KB blocks over time shows a pattern that is repeated again and again with other I/O activities, such as random writes.

DDRdrive vs OCZ Vertex EX 2

4KB Sequential writes after Secure Erase and over one hour's sustained testing.

The Vertex 2 EX initially writes at 50,000 IOPS but after six minutes or so the IOPS number falls off a cliff and drops to 13,000 - 15,000 IOPS where it stays for test after test from seven minutes to an hour after the test session started, and continues at a 12,000 - 10,000 IOPS level from 120 minutes to 180 minutes after the test started.

George sums up his findings like this: "Flash based SSDs succumb to dramatic write IOPS degradation in as short as 10 minutes after [the] device is unpackaged or Secure Erased."

OCZ: 'Not an apples to apples comparison'

OCZ says the Vertex EX 2 "uses Single-Level Cell (SLC) memory, which has 10 times the program/erase (P/E) cycles than MLC-based drives, and features superb 4k random writes up to 50,000 IOPS to give a performance edge over the previous generation."

The supplier says: "Rated speeds may vary slightly depending on the benchmark used, drivers, windows version, bios version and file size. We recommend using ATTO, IOMeter, and PC Mark Vantage for benchmarking SSDs and to achieve maximum rated specifications."

OCZ spokeswoman Jessica Luken said: "This really isn’t an 'apples to apples' comparison, and I know exactly why this company would compare its solution to SSDs. My boss put it into perspective with this analogy: 'It is like comparing a drag car designed to accelerate fast in straight lines to a Ferrari which may not be the fastest vehicle but is built to negotiate the track'."

Luken said: "It is no secret that one of the attributes of NAND-based storage devices is a change in performance from new, fresh-out-of-box (FOB) state to 'pre-conditioned' state. All NAND-based drives would exhibit similar results in this comparison ... The OCZ Vertex2 EX series of drives is among the best in the industry at maintaining close to FOB performance," she said.

"There are, of course, corner cases that show a larger drop in performance. These corner cases – when paired with a platform and software environment that is not NAND SSD optimised – can be made to appear more significant than real world scenarios. OCZ's FAE [Field Application Engineering] team works closely to enterprise customers to ensure their environment is optimised to maintain best-in-class performance and endurance."

She added: "I’d also argue that the benchmarks provided are not accurate, as the testing doesn’t even exhibit what all other third party media are reporting, and it’s clear the testing was definitely run on an unTRIMed drive in dirty state."

OCZ Field Applications engineer Mark Hayashida added: "Though it is true that performance of Flash-based drives can change from its 'new' condition (or after a Secure Erase) over time due to certain conditions, this [DDRdrive] presentation is only looking at a particular area that calls out some worst case scenarios for Flash-based drives, such as ZFS and its intent log (ZIL) performance...

"Typically (what I call an I/O Signature of) ZIL is usually spiky in nature, not always sustained, so with TRIM support, when it comes to ZFS, that should help performance on Flash SSDs running ZIL. That would provide time for the drive to do some 'housekeeping' and maintain a higher level of performance over the life of the drive."

'Specifications are knowingly disingenuous'

Regarding TRIM, a means for Windows to tell the SSD controller about deleted data in the SSD, DDRdrive's George said: "I did run the exact same tests on Windows 7 (which does support TRIM) and the OCZ results do marginally increase. But the overall trend line stays constant; *dramatic* IOPS degradation relative to published specifications. Basically, unless one uses a development version of Iometer, which happens to default to a contrived data pattern, the published specifications are knowingly disingenuous."

The OCZ pitch is that we should use the right SSD for the right job. The X1 may well be great at ZIL performance and the Vertex 2 EX less suitable, but that does not mean you can generalise from ZIL to everyday flash SSD application performance. In other words, you can't use an X1 dragster on the open road but you can use a Ferrari Vertex 2 EX. ®

High performance access to file storage

More from The Register

next story
European Court of Justice rips up Data Retention Directive
Rules 'interfering' measure to be 'invalid'
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Bored with trading oil and gold? Why not flog some CLOUD servers?
Chicago Mercantile Exchange plans cloud spot exchange
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
prev story


Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.