Can all-flash arrays, spinning disks and hybrid really live together?
AFA without tears
Conventional wisdom says that for performance-sensitive applications, you want all-flash arrays (AFAs), while for less-critical applications and bulk storage, disk is better. How can you use them in conjunction with each other? What would a joint AFA/hybrid/disk environment look like?
Broadly speaking, there are two types of flash array: the pure, all-flash kind that stores everything in a super-fast SSD; and then the hybrid array, which combines a relatively small proportion of flash with some spinning drives that provide more capacity. Then, of course, there’s your conventional spinning disk, which comes with its own RPM speeds. We typically understand the limitations of spinning disk, which revolves around input-outputs per second (IOPs). Disk IOPs usually run into the hundreds, while flash SSD registers in the tens of thousands.
Hybrid, of course, merges the two, and is supposed to provide the best of both worlds. One of the biggest advantages of hybrid drives overall is cost, point out experts. Given the still-high cost of flash memory, people don’t want to buy any more of the stuff than they must, points out Frank Berry, CEO at data centre infrastructure analyst firm IT Brand Pulse. “Everyone identifies their most frequently accessed data and stores that on flash,” he said.
Density in solid state storage is “zooming past” hard drives, in his estimation, but they’re still more expensive than HDDs, and most customers are way more sensitive about the cost per gigabyte. For this reason, the vast majority of solid state storage is still sold on a hybrid basis. IDC figures bear this out. “According to IDC's Worldwide Enterprise Storage tracker and Forecasts, the move towards flash-powered arrays in Western Europe is well under way: AFA accounted for 17 per cent of total revenue in 2016Q2, and HFA for an additional 46 per cent,” she said.
Where they use hybrid arrays, IT teams will prioritise the use of the SSD for frequently accessed files. One example is in databases, said Berry. “Within the databases there are index files that are accessed more so than in the data,” he says. “A typical solution would be to put index files on SSD drive in hybrid.” That way, the IT team doesn’t have to rely on expensive caching.
From hybrid to AFA
The balance between hybrid and AFA is slowly shifting, though, and databases are a good example of why. “A large database is in the 2-300Gb range,” said George Crump, founder and president of storage technology advisory firm Storage Switzerland, arguing that the falling cost of flash is making it more viable to put this entirely on an SSD. “And all that is before we factor in deduplication and compression,” he said. Databases may not be built for deduplication, but they decompress well, and this function normally happens inline with all-flash units.
By 2020, IDC expects 87 per cent of the total external storage market to represent flash-powered arrays, and 40 per cent of them will be all-flash. Often, hybrid flash will be a go-to choice among smaller to medium-sized customers who want better performance than pure spinning disk but simply aren’t ready to branch out into all flash arrays, points out Frank Reichart, senior director of product marketing at Fujitsu.
“In larger accounts or among data centre customers, all the production systems for mission-critical applications simply go on all-flash only, with the exception of online archives,” he says.
Jim Handy, general director at semiconductor market research firm Objective Analysis, agrees. While companies grapple with SSDs in general, hybrid arrays are typically finding their niche among smaller firms.
“Once you start having arrays of disks, then you don’t want to put the flash into the disks themselves. It’s less expensive to have a centralized SSD and then use software to manage the distribution of data from the disk to the SSD,” he said.
Every byte of flash storage that you buy from a hybrid vendor is more expensive than the same amount in an all-flash array, he argued. That makes hybrid still a good bet for smaller firms who are more cash constrained and can do without the large performance bump that an all-flash array can provide.
So what’s the point in using hybrid arrays at all? One answer is that they tier their data automatically. Smaller firms relying on a hybrid array for their entire backend system are unlikely to want to manage what data goes where. With a hybrid setup, they don’t have to – the controller takes care of all that under the hood.
This advantage is exacerbated by the variety of tiering options available in hybrid SSD products. Some offer a straight HDD/SSD split, although the proportion of each will vary. However, you can also get sub-tiers of SSD storage, each configured for read and write-intensive operations. And the hard drive layer can be tiered into different disk speeds, too. This creates a flexible array of configuration options for companies not yet ready to take the plunge into AFA.
As the industry does move to AFA, Simon Robinson, research veep at 451 Research, advises against trying to integrate them with hybrid SSD in any meaningful way. Although some firms like NetApp have been focusing on trying to make an all-flash version of a traditional hybrid SSD system, the two kinds of storage aren’t usually compatible, he warns.
“These are systems that run on entirely different operating systems. The way they store data, the way they dedupe data, the algorithms that they use, they’re all completely different,” he says, adding that this makes some idealistic visions of interplay between the two platforms difficult. Customers might want to use all-flash arrays as the primary system, and then send snapshots from that system to a hybrid one. In the real world, that’s not really possible, he says. It would take a lot of technical work to make it happen.
“My overriding problem in storage already is that I have too many silos. And you’re saying I need another silo because my flash array doesn’t integrate with the rest of my environment,” he says. For this reason, he argues that some companies will choose one or the other. Those that do use them together will find them uneasy bedfellows, and you’ll typically find administrators dividing AFA and hybrid along application-specific lines.