PCIe flash performance: Your mileage may vary
Researchers say TMS tops Fusion-io and Virident
A Swiss supercomputing centre has found TMS PCIe flash delivers its advertised goods while Fusion-io and Virident do not.
Two researchers at the Swiss National Computing Centre (CSCS) looked at the performance of PCIe flash cards from Fusio-io and Virident (pdf) and Texas Memory Systems (PDF).
The tests used the open-source FIO and IOR benchmark tests: FIO for "benchmarking specific disk IO workloads" and IOR "to benchmark parallel file systems". The tested cards were all single-level cell (SLC) PCIe cards.
The first test looked at a Fusion-io ioDrive Duo with 320GB of SLC flash and advertised peak IOPS of 260,000 and 1.5GB/sec. The second card was a Virident TachION with 400GB of flash, which is supposed to run at 300,000 IOPS – 1.4GB/sec when reading and 1.2GB/sec when writing.
Both cards are advertised with significantly higher specs than observed in our environment.
In general, the ioDrive Duo delivered up to 145,000 write IOPS and 135,000 read IOPS, rather less than advertised. Its bandwidth was up to 1.5GB/sec but only with large block sizes. It was 400 to 600MB/sec when writing 4KB blocks, again less than the peak figure, far less.
There were variations between the two tests and variations depending on block size. Refer to the PDF file for more details.
The Virident card delivered up to 200,000 read IOPs and 160,000 write IOPS, again less than the 300,000 rated IOPS. Its bandwidth was pretty close to the advertised figure: 1.2GB/sec when writing and 1GB/sec when reading. Virident reckoned there could be a bug in its driver code.
There were variations between the two tests and variations depending on block size. Refer to the PDF file for more details.
The researchers concluded: "Both cards are advertised with significantly higher specs than observed in our environment. We believe that our analysis tools give a more realistic view of what one can expect in a real production environment."
RamSan results
In a second test, the researchers looked at the RamSan-20 - a 450GB card - and the newer RamSan-70, also with 450GB, half its maximum capacity. We're interested in the RamSan-70 results. The researchers reckoned it would do half the rated numbers for a 900GB product, that is 300,000 IOPS and 1GB/sec.
It delivered 300,000 IOPS and 1.1GB/sec. The researchers noted: "The peak read bandwidth of 1.1GB/s is provided through all block sizes ... which is very different from what we reported for the Fusion-io and the Virident devices... The RamSan-70 provided by far the best IOPS result we have ever measured at CSCS with a peak of more than 300,000 IOPS at 4K for FIO ... which is consistent to the expected value... [it delivered] a very stable IO bandwidth over a wide range of block sizes which is a clear differentiator to the cards tested in the previous study."
The RamSan-20 also performed as the researchers expected.
The CSCS results are unequivocal. Check the PCIe flash cards you are thinking of buying against your own workloads as manufacturers' rated numbers may not match what you will see in your own shop. ®
COMMENTS
Dissimilar systems used for benchmarking
An astute reader will notice that two different servers were used by CSCS in their testing. For the Fusion and Virident testing, they used a system with an AMD 2.0Ghz processor. For the RamSAN testing, they used a 2.93Ghz Intel processor.
That's not apples to apples, so it's quite questionable to compare the results between the two tests.
Fly in the ointment?
It is perfectly valid to run a different workload than what is spec'd by vendors, and report the results of this test. It is, however, grossly unfair to report that a product does not "deliver its advertised goods" when the advertisement is for apples, and the comparison made was to oranges.
It's worth noting that the that the CSCS folks did not duplicate workloads quoted by the vendors in any of the cases. Fusion quotes for 512B IOs, not 4k. Virident quotes for a 75%/25% read/write workload- not the read only and write-only workloads tested in this paper. No flash vendor I've encountered specifies numbers for performance using XFS.
Not in the review,
but I have worked with TMS before. They are good guys, vary willing to support their product.
(I am in no way, shape, or form affiliated with the researchers or author of this article)
IT infrastructure monitoring strategies
What you need to know about cloud backup
Enabling efficient data center monitoring
Agentless Backup is Not a Myth
Top 10 SIEM Implementer’s Checklist
