The future of storage: disk-based or just discombobulated?
You want disk tech innovations? We got 'em, lots of 'em.
The disk drive industry went bananas in 2013, driven hard by cloud storage and flash, both sending the problems of bulk, nearline data storage disks' way.
The three big HDD tech issues?
- SMR technology seems a near-term bust except for big cloud users
- Helium drive technology arrived with HGST's 6TB offering
- Seagate announced its Ethernet interface Kinetic drives
The 3.5-inch performance disk drive business is perceived of as being finished. Hot data will increasingly come out of flash cells and not off disk. Disks are unrivalled for cheap and fast enough online storage. In the 2.5-inch area there is still a need for fast drives of 10,000 rpm but this may be a short-lived phenomenon.
The general situation is:
- Proportionately more bulk data storage will be done in public clouds, with million-plus spindle estates
- On-premise data centres will also need bulk, online disk storage, with RAID rebuild time a continuing problem
- NAS arrays for small business are booming
- Object storage is a feasible technology for scalable, self-healing disk data stores seen through a file access presentation layer
- Desktop disk drive sales are falling as tablet computing gets more popular and tablet/notebook/desktop SSD use rises
These trends have resulted in Seagate's Kinetic drives, two vast shingled drive trials, and WD's re-invention of hybrid flash/disk drives. HGST's late 2012 revelation of its Helium-filled drives was followed by a product announcement in the second half of 2013. We have also seen acquisitions by the three disk-drive manufacturers; Seagate-Samsung, WD-HGST, and Toshiba as they diversify into flash and also move up the disk storage stack from their disk drive and consumer/SME 1-8 drive slot array base, heading towards storage arrays.
Helium drive technology
Filling a disk drive enclosure with helium, a much lower friction environment than plain old air, means more platters and heads can be put inside a drive enclosure, thus increasing the drive's capacity. The drive industry had looked at this in the past but helium will leak through the smallest holes and this difficulty had helped prevent the technology become production-ready.
HGST worked out how to hermetically seal the drive effectively, and up to 7 platters can be stuffed inside a traditional 3.5-inch disk drive enclosure. Currently 4-platter 3.5-inchers hold 4TB, 1TB/platter. Thus a 7-platter drive could hold 7TB.
In the event HGST launched a 6-platter, 6TB Ultrastar He6 in November, giving it outright capacity leadership in the 3.5-inch drive space. Seagate is answering this with an anticipated 5TB or 6TB shingled magnetic recording (SMR) media drive early this year. It will, however, have the known SMR drawback of slow data rewrites, where a band of tracks have to be rewritten upstream of a data replacement exercise because of SMR's track overlapping.
It looks like HGST has an edge here.
Small form factor drive action
The drive manufacturers built hybrid and thin 2.5-inch drives for tablet and notebook use, aiming to add cheaper and denser capacity, with flash cache acceleration, and so provide an alternative to more expensive all-flash storage. The provision of 2.5-inch drives as 3.5-inch drive replacements in small form-factor desktops and notebooks also progressed throughout the year.
More extreme 2.5-inch drives came along, such as HGST's 3-platter 1.5TB drive inside a standard 9.5mm-thick 2.5in form factor. Imagine the possible platter counts in helium-filled 2.5inch drives; four platters ought surely to be possible.
Western Digital announced a radically thin single platter 2.5-inch drive, the UltraSlim just 5mm thick.
Seagate dropped its 7,200rpm 2.5-inch drives in favour of 5,400rpm hybrid ones with data access sped by a NAND cache in March, and in June moved its hybrid notebook tech into the enterprise with a 600GB Savvio spinning at 10,000rpm and having a 16GB NAND cache. IBM signed up to use the drive.
Toshiba introduced a single platter 2.5inch hybrid in June, offering 320GB and 500GB capacity in a 7mm thick case.
Vendors catch shingles
In February Seagate said it would ship SMR drives later in the year. It didn't. Instead it revealed in an earnings report in September that it had shipped a million SMR drives to an unrevealed customer or customers. (EVault - see below.)
SMR drives cram tracks closer together by overlapping groups of them, leaving valid read tracks inside the boundaries of the overlapped and wider write tracks. SMR drives can store more data than existing perpendicular magnetic recording (PMR) drive technology drives but data re-writes are much slower, as all affected overlapped tracks have to be sequentially re-written.
SMR drives also seem much more sensitive to vibration, meaning that they either have to be packed less densely in arrays or used in spin-down arrays where the majority of drives are inactive.
Seagate joined the OpenStack Foundation and the Open Compute Project in February "to help foster the growth of cloud storage solutions." Ali Fenn, a senior director at Seagate, said; ""We're not going to open source the internals of [our] drives, but we'll be looking to work with the communities' hardware and software players. We recognise that these are the stacks of the future." For Seagate cloud storage of data is becoming on of the most important technology direction drivers.
Towards the end of the year we saw one of the fruits of this when Seagate's EVault cloud backup subsidiary announced in December its Long-Term Storage Service (LTS2) for storing archival data on spun-down disk in the EVault cloud. We suspect this service uses Seagate's SMR drives but nobody is saying.
After having talked about SMR technology in June, WD revealed that Facebook is trialling its SMR drives for storing photos in an online disk archive, with faster photo access than tape can provide. This is Facebook's cold storage idea.
So the net of it is that both Seagate and WD have shipped around a million SMR drives each to low numbers of customers and have not made SMR drives generally available, despite their seeming lower cost/GB. Why is that?
We surmise two things; re-write performance is dreadful and vibration is a killer. These prevent the drives being suitable for general enterprise and SME use, maybe for a good few months yet and maybe even forever, since alternative drive capacity-increasing technologies, such as helium-filling and HAMR (Heat-Assisted Magnetic Recording) are here or close and will render SMR drives redundant except for the specific online disk archive with fast data access applications exemplified by Facebook's use of the tech for storing old photos and EVault's faster than (tape-based) Amazon's Glacier cloud archive at retrieving data.
HAMR drives are still in the pre-announcement phase with WD's CEO giving a presentation off a prototype HAMR drive in November, more or less a year after Seagate's CEO pulled the same trick. Still no product though.
Let's get Kinetic
Seagate introduced its direct access Kinetic Drive technology in October, with a 4TB drive. These drives are accessed over Ethernet by applications, and have object storage style access using key:value stores. Seagate said; "The majority of today’s mass scale object applications do not need either file semantics or a file system to determine and maintain the best strategy for space management on a device."
It said the total cost of ownership for a disk-based storage infrastructure could be lowered by up to 50 per cent for cloud storage service providers with such drives. In effect the drive space management functionality provided by a storage array controller is divided between the drives, which are more intelligent, and server system software.
The announcement including supporting quotes from Basho Technologies, Dell, EVault, Huawei, Hyve, Rackspace, Sanmina (Newisys division), Supermicro, SwiftStack, Yahoo and Xyratex. Note that last company's involvement.
If either WD or Toshiba support the Ethernet carried object style access then Kinetic drive technology has a more assured future. We watch and wait.
Reinventing hybrid drives
WD went and reinvented hybrid drive technology with its Black Dual Drive laptop spinner. This featured a vastly larger slug of flash than other hybrid drives, at 120GB, and presented it as a second logical drive even though it was in the same 2.5-inch enclosure as a 1TB spinning disk drive. This drive theoretically provides flash speed for the host operating system and applications while also providing a terabyte of cheap disk capacity for data; the best of both flash and disk worlds.
Will Seagate and Toshiba follow suit? They are not saying. Once again, we watch and wait.
The product's flash and disk separation marked it out as different from Apple's Fusion drive which presents two physically separate drives, one flash and one disk, as a single logical drive.
Acquisitions and investments
Business was brisk on the M&A front. Here's a 2013 timeline:
- January - WD buys Arkeia for its backup software and appliance business
- January - Seagate invests $40m in PCIe flash HW/SW startup Virident
- March - Western Digital invested in all-flash array startup Skyera's second funding round
- June - Western Digital buys fallen enterprise SSD star sTec and sticks it in its HGST subsidiary
- July - Western Digital buys flash cache software startup VeloBit and assigns it to its HGST sub
- August - Toshiba invests in Zadara which is developing technology for virtual private storage arrays in the cloud. It said; "The two companies will be working together to introduce innovative Storage-as-a-Service (STaaS) products using cutting-edge technologies they have each developed."
- August - Seagate invests in eASIC to add new interfaces to its drives
- September - WD buys Virident and gives it to, yes, its HGST subsidiary which handles nearly all matters flash for WD (except for hybrid flash/disk drives). Seagate rejected a chance to buy Virident
- December - Toshiba buys the assets of crashed SSD star OCZ
- December - Seagate buys Xyratex for its HDD test equipment, its OEM disk array enclosure, and its ClusterStor HPC array businesses
On the financial front Seagate is looking more likely to get a $630m boost to its coffers courtesy of an arbitration award over WD recruiting on of its execs and allegedly using secret Seagate information to develop new read/write head technology. This had risen to $706m in October.
It seems noteworthy that with Helium-filled drives from HGST and Kinetic Drives from Seagate there is no second source. It used to be a rule in enterprise disk drive purchases that you never restricted yourself to single source disk drive technology. That idea seems to no longer apply, rendering customers for such drives utterly dependent on one supplier. The benefits of new technology, such as Kinetic or SMR drives, outweighs the single source risk for the large customers involved.
In 2013 we saw the three disk drive manufacturers and their strategies diverging. WD is going after flash products more determinedly while bringing out its SMR tech and HGST its Helium-filled drive products. Seagate is building disk-based products for the cloud and these are taking precedence over its flash storage activities, for now.
In 2014 we should see if SMR drives do make it to general availability. We should also see if the Chinese industry regulator permits Seagate and Samsung on the one hand and WD and HGST on the other to merge their operations. This would enable both Seagate and WD to rationalise their production, research and development and support activities and become more efficient.
The direction and scope of WD/HGST's NAND storage product technology directions should become more visible, as should those of Seagate and Toshiba. Will WD and Toshiba bring out their version of Kinetic Drives? Will Seagate embrace helium-filled drive technology and WD's logically separate flash/disk hybrid drive ideas? How will Toshiba use its OCZ SSD and controller SW assets? What will Seagate do with the three Xyratex businesses?
Questions, questions, questions; Vulture Central's storage desk intends to bring you the skinny on all the disk drive industry's developments in 2014. ®