Quick as a flash: A quick look at IBM FlashSystem
Texas Memory memories...
The Storage Architect IBM completed the acquisition of TMS (Texas Memory Systems) in October 2012. The company (TMS) has a long history of developing memory-based storage products, based initially on DRAM and later SLC and eMLC technology. I looked at the ramsan range years ago as a potential customer and my abiding memory is the relative cost of the hardware compared to today’s all flash systems. However, that was at a time before EMC had started to introduce SLC drives into VMAX and Violin Memory was only a twinkle in the founder’s eye.
The TMS acquisition forms the foundation of IBM’s FlashSystem products. There are two main lines: the 900 Series, which is a flat out dragster product and the V9000, which combines 900 series appliances and IBM’s SVC (SAN Volume Controller) to provide the scalability and features that aren’t present in the standard appliance.
Looking at the speeds and feeds, the FlashSystem 900 claims latency numbers as low as 90µs (write) and 155µs (read), although these are minimum rather than typical values. Throughput is a claimed 1.1m IOPS (100 per cent random read) with 4KB block size and 600,000 IOPS (100 per cent random write) or around 800,000 IOPS with a 70/30 per cent read/write split.
Bandwidth is quoted at 10GB/s (100 per cent sequential read) and 4.5GB/s (100 per cent sequential write), so a more balanced workload probably sits somewhere in between as 100 per cent sequential isn’t typical in anyone’s book. For an all-flash product, these are good numbers. However, we have to remember that with the exception of RAID-5 support, there are no other native features in the 900 Series platform. A single 900 Series array is limited to around 105TB in capacity (raw) or 57TB usable after RAID-5 overhead is taken into account.
FlashSystem 900 uses custom-made flash media, branded by IBM as MicroLatency modules. These use a mix of 20nm MLC flash NAND, FPGAs and software IP deliver a bespoke storage medium. This direction is also being followed by Violin Memory with VIMMs, Pure Storage with FlashBlade and Hitachi with FMDs.
FlashSystem and SVC
IBM has chosen to add features into the FlashSystem range by combining SVC (SAN Volume Controller) with 900 Series appliances. SVC is a dual-controller architecture based on 2U form factor. It uses dual Xeon E5 processors, 64GB of memory and supports Fibre Channel, Ethernet (iSCSI) and FCoE connectivity. So an entry-level or building block (BB) V9000 system consists of two controllers (AC2 units) and an AE2 storage enclosure (basically a 900 Series), delivering a maximum of 57TB (usable) in 6U.
By way of comparison, Violin Memory’s 7300 platform can get 70TB in 3U, or 44.3TB usable, nearly double V9000’s density. This isn’t exactly earth shattering in terms of capacity or density, however multiple BB systems can be combined together, based on the limits of SVC, allowing a cluster of up to four BBs (8 AC2s, 4 AE2s). Additionally, capacity can be further expanded through the addition of up to four extra AE2 storage controllers. All of this configuration is connected through the use of a private 16Gb/s Fibre Channel network.
A maximum configuration of four building blocks and four storage enclosures does seem a little top-heavy, with 50 per cent of the rack space assigned to controllers. This is an effective maximum of 14.25TB of capacity per rack unit. Obviously if throughput/performance isn’t the issue then a single BB can be deployed with an additional four storage controllers, which significantly improves the capacity per rack unit ratio.
So what’s the impact of putting SVC in the data path?
Well a single BB delivers 200µs latency (minimum), up from 155µs – write performance strangely isn’t quoted – however this is still a good figure. Throughput suffers significantly though, from 1.1m IOPS to only 630,000 IOPS (read). With Real-Time Compression (RTC) enabled, IOPS are halved again. Incidentally these are numbers that Violin’s FSP7300 can happily equal, although most other vendors are a little way behind.
SVC provides the advanced features to V9000, including compression (through additional hardware add-in cards), thin provisioning, snapshots, clones and replication. One notable omission is data de-duplication, which isn’t available on SVC. Real time compression does achieve a claimed 5:1 reduction on “relevant” data sets, however de-duplication (dedupe) can provide significant savings over and above what compression is able to achieve. I wonder if the ability to deliver dedupe is compromised by using the SVC architecture to deliver services.
SVC is a stateless in-band virtualisation layer and so holds no configuration data other than the LUN/volume to storage mapping. Implementing the metadata needed to track de-duplicated data may require significantly more DRAM and additional persistent storage than is currently in place within SVC. In addition, the dedupe functionality would have to be federated across multiple SVC node pairs in a cluster to see the greatest capacity savings. Managing this would introduce additional latency overhead and compromise some of the benefits of V9000.
The Architect’s View
FlashSystem 900 is definitely a super-fast platform if your application needs low latency and high bandwidth. However there are new challengers in the market for low latency storage, such as EMC DSSD and Mangstor, which could threaten this end of the business. The more feature rich V9000 is still a fast product, but depends on the legacy of SVC, which has some significant shortcomings like lack of dedupe and overall space efficiency. I think the question to ask here is what is next for FlashSystem.
Other vendors are tackling the relative cost of flash (compared to HDDs) with new technologies like TLC and 3D NAND. Adoption of these chips would mean building new MicroLatency modules, which shouldn’t be a challenge for IBM (they upgraded from 24nm to 20nm process chips with the move from the previous V840 to V9000). However today modules within a single AE2 cannot be mixed; deploying an entire storage controller to get larger drive modules does seem to be overkill.
I like the TMS technology, but as you can probably tell, I don’t like SVC. It would be nice to see IBM pushing some of the more advanced functionality into the 900 Series (or whatever comes next) and removing the dependency on SVC wherever possible. With an announcement due soon, perhaps we’ll see some of that appearing.
Note: I don’t get briefed by IBM on FlashSystem, either current or future products, so all of my assumptions here are based on publicly available documentation. Please feel free to correct where necessary.