Feeds

Big Blue boffins scan 10 billion files in Flash in a flash

Enticing developments at the lab

Choosing a cloud hosting partner with confidence

IBM and Violin have announced a great big GPFS numbers record: the software scanned 10 billion files in a flash – well, 43 minutes – using four Violin flash memory arrays.

This was 37 times faster than a previous GPFS record of scanning one billion files in three hours, but that was with the file system metadata stored, like the file data, on disk drives.

Why does this matter? IBM says it is because GPFS needs to scan files its filesystem so that they can be moved between storage tiers, migrated, archived, etc. This is non-production work and has to be done in the background. When done with metadata on disk, the process becomes slower and slower as the number of files in a GPFS system rises and rises. So much so that, conceivably, eventually there aren't enough hours in the day to do it.

So IBM Research tried putting the metadata on flash arrays and seeing how much faster the system went. The result is impressive, very impressive, but not that surprising. It's also a tad, well, background, since the system wasn't handling real data.

Back in February, IBM announced a wondrous SONAS SPECsfs 2008 benchmark result of 403,326 SPECsfs ops/sec from a single GPFS system using 1,975 hard disk drives.

EMC bounded past this with a flash-heavy VNX system doing 497,623 ops/sec, using 436 x 200GB, SAS SSDs and 8 file systems.

SONAS is based on GPFS. From where El Reg sits, a re-run of the IBM SONAS SPECsfs2008 benchmark looks feasible, but this time using a few Violin Memory Arrays to hold the SONAS data and so get to the 500,000 SPECsfs2008 ops/sec area and beyond. We have asked both IBM and Violin about this but didn't expect to get anything looking like a "Yes, we're doing this" reply.

Much to our surprise we received this from Bruce Hillsberg, director of storage systems, IBM Research – Almaden: "You are correct: if we were to re-run the SPECsfs benchmark on a SONAS system with using the technology described in the press release, we would see a significant performance improvement."

Enticing, isn't it? ®

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
'Kim Kardashian snaps naked selfies with a BLACKBERRY'. *Twitterati gasps*
More alleged private, nude celeb pics appear online
Wanna keep your data for 1,000 YEARS? No? Hard luck, HDS wants you to anyway
Combine Blu-ray and M-DISC and you get this monster
US boffins demo 'twisted radio' mux
OAM takes wireless signals to 32 Gbps
Google+ GOING, GOING ... ? Newbie Gmailers no longer forced into mandatory ID slurp
Mountain View distances itself from lame 'network thingy'
Apple flops out 2FA for iCloud in bid to stop future nude selfie leaks
Millions of 4chan users howl with laughter as Cupertino slams stable door
Students playing with impressive racks? Yes, it's cluster comp time
The most comprehensive coverage the world has ever seen. Ever
Run little spreadsheet, run! IBM's Watson is coming to gobble you up
Big Blue's big super's big appetite for big data in big clouds for big analytics
Seagate's triple-headed Cerberus could SAVE the DISK WORLD
... and possibly bring us even more HAMR time. Yay!
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Beginner's guide to SSL certificates
De-mystify the technology involved and give you the information you need to make the best decision when considering your online security options.