Feeds

Microsoft flash research inspires EMC

Data Domain flash-based dedupe coming

Internet Security Threat Report 2014

EMC is using Microsoft research into flash use as a memory tier between DRAM and disk drives to speed up Data Domain deduplication.

FlashStore is a Microsoft research project that batches up flash writes in a server's main memory and then writes them as a page or block, in effect turning random flash writes into sequential ones and avoiding block erase/write cycles that are otherwise endemic to flash.

Such cycles both slow down flash writes and can shorten its working life by increasing the number of block erase/write cycles in a time period, flash blocks having a finite erase/write cycle capacity.

The Microsoft researchers Sudipta Sengupta, Jin Li and intern Biplob Debnath said FlashStore minimises its DRAM footprint by using an index to access the data in the flash cache.

This is a key:value store using a hash-table index. Microsoft tells us it saves space on both the vertical and horizontal dimensions of the table by using a version of cuckoo hashing - where keys in table slots serve as eggs pushed out of the nest - to reduce the number of slots and slot sizes in the hash table.

In cuckoo hashing, as we understand it, for any key value two hash functions are used instead of one, giving the key two possible locations in the table instead of one. Any pre-resident key in that slot is booted out and it is then put in its alternative slot, booting out any already resident key until an empty slot is found.

Apparently this saves up on lookup time, with just two slot inspections needed. But the table has to be less than half full and, if it gets full, it has to be rebuilt because there isn't an empty slot for the last displaced key value hash.

Returning from computer science territory

The researchers say that the FlashStore DRAM table is frugal in its use of DRAM and provides fast access time with an average of a single flash read per lookup. This means servers using it can perform many times faster; "several tens of factors of increased data throughput" is the phrasing they use.

Sengupta says this about applications with high IOPS needs: "You could replace 10 to 20 hard drives with one flash drive using FlashStore for such applications.

"That gives you a capital-expenditure savings, power savings, and operational-expenditure savings, as well, and you also get much faster throughput: an all-win situation on the three metrics of price, power, and performance.”

Deduplication

It sounds like deduplication could use this technique. Microsoft's FlashStore paper says: "FlashStore also achieved significantly better results than a RAM/hard drive or a flash drive when tested with the task of performing data-chunk indexing for data deduplication."

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
Azure TITSUP caused by INFINITE LOOP
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
You think the CLOUD's insecure? It's BETTER than UK.GOV's DATA CENTRES
We don't even know where some of them ARE – Maude
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Stop the IoT revolution! We need to figure out packet sizes first
Researchers test 802.15.4 and find we know nuh-think! about large scale sensor network ops
DEATH by COMMENTS: WordPress XSS vuln is BIGGEST for YEARS
Trio of XSS turns attackers into admins
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Seattle children’s accelerates Citrix login times by 500% with cross-tier insight
Seattle Children’s is a leading research hospital with a large and growing Citrix XenDesktop deployment. See how they used ExtraHop to accelerate launch times.
5 critical considerations for enterprise cloud backup
Key considerations when evaluating cloud backup solutions to ensure adequate protection security and availability of enterprise data.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?