Feeds

Could a hard drive dedupe data?

Manufacturers start looking for the next big thing

Beginner's guide to SSL certificates

Comment Hitachi GST president Steve Milligan says one of the drivers affecting the hard drive industry is the need for efficient storage with technologies like virtualisation and deduplication. What is he on about?

He presented at a recent Needham conference for HDD investors and said that the storage market was driven by three things. First was the need to store more data, which he saw continuing if not accelerating. Then he said mobility had risen strongly and was set to continue as people needed to store more data in mobile devices. Thirdly, he mentioned storage efficiency and virtualisation and deduplication.

Why would a hard drive manufacturer be bothered about these two items? All HGST and the other drive suppliers do is build a raw byte storing device and present it to drive array and compute device manufacturers as a component. They add virtualisation functions and may build the drives into deduplicating storage arrays. Deduplication simply isn't a concern of drive manufacturers, or is it?

A couple of years ago, data encryption was performed by in-band devices, such as NetApp's Decru boxes, applications running in a server, or tape drives that encrypted backup data in case the tape cartridges got lost or stolen.

As encryption hardware was then small enough to shove in a tape drive controller, it was also small enough to put inside a hard drive's case. So Seagate and others came out with self-contained hard drive encryption products.

Could Milligan be thinking that where an encryption ASIC can go then a deduplication ASIC could follow?

You would need to do HDD-level dedupe inside the HDD case, where space is restricted, and you'd need to provide CPU cycles because the HDD controller CPU is already busy doing standard HDD I/O-type tasks. Suppose there were an ASIC, an application-specific processor you could use?

Exar has bought Hifn for its hardware encryption, compression and deduplication products. Here is a description of a Hifn chip:

The 9725 provides power efficient hardware offload capabilities for data compression, encryption and deduplication for enterprise class storage applications. OEMs now have the ability to integrate these capabilities into platforms that require non-standard PCIe form factors or need to be embedded directly on a system’s motherboard.

Solid State Drive start-up Whiptail is adding Hifn dedupe to its RaceRunner SSDs, so the idea isn't entirely half-arsed.

As storage array futures look to be heading to two main disk tiers, one for performance (FC, SAS, SSD) and one for capacity (SATA) then taking a 2TB SATA drive used for virtual tape library and disk-to-disk backup applications and increasing its capacity by 20, 30 or possibly more per cent by deduplicating data stored on it, with no decrease in I/O performance, might look interesting to a hard drive manufacturer looking to gain an edge and justify a pricing differential.

By the way, for a vendor of sealed disk drive canisters, aka Integrated Storage Elements (ISEs), like Xiotech, adding this functionality at a canister level might be worth considering.

Of course Milligan could just have been talking generally. But it's enticing isn't it? Once HDDs didn't encrypt their data. Now they do because there is enough HDD controller board real estate to permit the addition of encryption hardware. What else might be added, what other functions might be considered by the HDD suppliers? Could on-board - or in-canister - HDD deduplication be a realistic prospect over the next few years? ®

Security for virtualized datacentres

More from The Register

next story
It's Big, it's Blue... it's simply FABLESS! IBM's chip-free future
Or why the reversal of globalisation ain't gonna 'appen
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
CAGE MATCH: Microsoft, Dell open co-located bit barns in Oz
Whole new species of XaaS spawning in the antipodes
Microsoft and Dell’s cloud in a box: Instant Azure for the data centre
A less painful way to run Microsoft’s private cloud
AWS pulls desktop-as-a-service from the PC
Support for PCoIP protocol means zero clients can run cloudy desktops
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.