Sun setting dedupe up for ZFS
Australian conference giveaway
Oracle/Sun's ZFS file system seems set to get deduplication added to it later this year.
ZFS or the Zettabyte File System is a 128-bit file system that Sun says radically simplifies file system administration. Amongst its features are 256-bit checksums that detect and correct silent data corruption. In other words, it is already looking at data blocks inside the file system and checking their integrity. This could be extended to look for duplicated blocks or groups of blocks.
A thread on the OpenSolaris.org website mentions ZFS deduplication. Responses indicate that a synchronous version is being developed, one that works as data comes in. A background or asynchronous mode is also mentioned.
Later this month there will be a Kernel Conference in Brisbane, Australia, running from July 15 to July 17. The agenda includes a "Deduplication in ZFS" opening keynote, presented by Jeff Bonwick and Bill Moore. Jeff Bonwick is a Sun Fellow and VP, and the chief architect of ZFS. Bill Moore is a Distinguished Engineer at Sun and co-led with Jeff Bonwick on ZFS.
That seems pretty solid. ®