Feeds

Could a hard drive dedupe data?

Manufacturers start looking for the next big thing

Application security programs and practises

Comment Hitachi GST president Steve Milligan says one of the drivers affecting the hard drive industry is the need for efficient storage with technologies like virtualisation and deduplication. What is he on about?

He presented at a recent Needham conference for HDD investors and said that the storage market was driven by three things. First was the need to store more data, which he saw continuing if not accelerating. Then he said mobility had risen strongly and was set to continue as people needed to store more data in mobile devices. Thirdly, he mentioned storage efficiency and virtualisation and deduplication.

Why would a hard drive manufacturer be bothered about these two items? All HGST and the other drive suppliers do is build a raw byte storing device and present it to drive array and compute device manufacturers as a component. They add virtualisation functions and may build the drives into deduplicating storage arrays. Deduplication simply isn't a concern of drive manufacturers, or is it?

A couple of years ago, data encryption was performed by in-band devices, such as NetApp's Decru boxes, applications running in a server, or tape drives that encrypted backup data in case the tape cartridges got lost or stolen.

As encryption hardware was then small enough to shove in a tape drive controller, it was also small enough to put inside a hard drive's case. So Seagate and others came out with self-contained hard drive encryption products.

Could Milligan be thinking that where an encryption ASIC can go then a deduplication ASIC could follow?

You would need to do HDD-level dedupe inside the HDD case, where space is restricted, and you'd need to provide CPU cycles because the HDD controller CPU is already busy doing standard HDD I/O-type tasks. Suppose there were an ASIC, an application-specific processor you could use?

Exar has bought Hifn for its hardware encryption, compression and deduplication products. Here is a description of a Hifn chip:

The 9725 provides power efficient hardware offload capabilities for data compression, encryption and deduplication for enterprise class storage applications. OEMs now have the ability to integrate these capabilities into platforms that require non-standard PCIe form factors or need to be embedded directly on a system’s motherboard.

Solid State Drive start-up Whiptail is adding Hifn dedupe to its RaceRunner SSDs, so the idea isn't entirely half-arsed.

As storage array futures look to be heading to two main disk tiers, one for performance (FC, SAS, SSD) and one for capacity (SATA) then taking a 2TB SATA drive used for virtual tape library and disk-to-disk backup applications and increasing its capacity by 20, 30 or possibly more per cent by deduplicating data stored on it, with no decrease in I/O performance, might look interesting to a hard drive manufacturer looking to gain an edge and justify a pricing differential.

By the way, for a vendor of sealed disk drive canisters, aka Integrated Storage Elements (ISEs), like Xiotech, adding this functionality at a canister level might be worth considering.

Of course Milligan could just have been talking generally. But it's enticing isn't it? Once HDDs didn't encrypt their data. Now they do because there is enough HDD controller board real estate to permit the addition of encryption hardware. What else might be added, what other functions might be considered by the HDD suppliers? Could on-board - or in-canister - HDD deduplication be a realistic prospect over the next few years? ®

Bridging the IT gap between rising business demands and ageing tools

More from The Register

next story
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Attack of the clones: Oracle's latest Red Hat Linux lookalike arrives
Oracle's Linux boss says Larry's Linux isn't just for Oracle apps anymore
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
prev story

Whitepapers

Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.