Feeds

Could a hard drive dedupe data?

Manufacturers start looking for the next big thing

Internet Security Threat Report 2014

Comment Hitachi GST president Steve Milligan says one of the drivers affecting the hard drive industry is the need for efficient storage with technologies like virtualisation and deduplication. What is he on about?

He presented at a recent Needham conference for HDD investors and said that the storage market was driven by three things. First was the need to store more data, which he saw continuing if not accelerating. Then he said mobility had risen strongly and was set to continue as people needed to store more data in mobile devices. Thirdly, he mentioned storage efficiency and virtualisation and deduplication.

Why would a hard drive manufacturer be bothered about these two items? All HGST and the other drive suppliers do is build a raw byte storing device and present it to drive array and compute device manufacturers as a component. They add virtualisation functions and may build the drives into deduplicating storage arrays. Deduplication simply isn't a concern of drive manufacturers, or is it?

A couple of years ago, data encryption was performed by in-band devices, such as NetApp's Decru boxes, applications running in a server, or tape drives that encrypted backup data in case the tape cartridges got lost or stolen.

As encryption hardware was then small enough to shove in a tape drive controller, it was also small enough to put inside a hard drive's case. So Seagate and others came out with self-contained hard drive encryption products.

Could Milligan be thinking that where an encryption ASIC can go then a deduplication ASIC could follow?

You would need to do HDD-level dedupe inside the HDD case, where space is restricted, and you'd need to provide CPU cycles because the HDD controller CPU is already busy doing standard HDD I/O-type tasks. Suppose there were an ASIC, an application-specific processor you could use?

Exar has bought Hifn for its hardware encryption, compression and deduplication products. Here is a description of a Hifn chip:

The 9725 provides power efficient hardware offload capabilities for data compression, encryption and deduplication for enterprise class storage applications. OEMs now have the ability to integrate these capabilities into platforms that require non-standard PCIe form factors or need to be embedded directly on a system’s motherboard.

Solid State Drive start-up Whiptail is adding Hifn dedupe to its RaceRunner SSDs, so the idea isn't entirely half-arsed.

As storage array futures look to be heading to two main disk tiers, one for performance (FC, SAS, SSD) and one for capacity (SATA) then taking a 2TB SATA drive used for virtual tape library and disk-to-disk backup applications and increasing its capacity by 20, 30 or possibly more per cent by deduplicating data stored on it, with no decrease in I/O performance, might look interesting to a hard drive manufacturer looking to gain an edge and justify a pricing differential.

By the way, for a vendor of sealed disk drive canisters, aka Integrated Storage Elements (ISEs), like Xiotech, adding this functionality at a canister level might be worth considering.

Of course Milligan could just have been talking generally. But it's enticing isn't it? Once HDDs didn't encrypt their data. Now they do because there is enough HDD controller board real estate to permit the addition of encryption hardware. What else might be added, what other functions might be considered by the HDD suppliers? Could on-board - or in-canister - HDD deduplication be a realistic prospect over the next few years? ®

Internet Security Threat Report 2014

More from The Register

next story
Azure TITSUP caused by INFINITE LOOP
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
You think the CLOUD's insecure? It's BETTER than UK.GOV's DATA CENTRES
We don't even know where some of them ARE – Maude
DEATH by COMMENTS: WordPress XSS vuln is BIGGEST for YEARS
Trio of XSS turns attackers into admins
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
Astro-boffins start opening universe simulation data
Got a supercomputer? Want to simulate a universe? Here you go
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
5 critical considerations for enterprise cloud backup
Key considerations when evaluating cloud backup solutions to ensure adequate protection security and availability of enterprise data.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
How to simplify SSL certificate management
Simple steps to take control of SSL certificates across the enterprise, and recommendations centralizing certificate management throughout their lifecycle.