Feeds

Big Blue boffins scan 10 billion files in Flash in a flash

Enticing developments at the lab

Combat fraud and increase customer satisfaction

IBM and Violin have announced a great big GPFS numbers record: the software scanned 10 billion files in a flash – well, 43 minutes – using four Violin flash memory arrays.

This was 37 times faster than a previous GPFS record of scanning one billion files in three hours, but that was with the file system metadata stored, like the file data, on disk drives.

Why does this matter? IBM says it is because GPFS needs to scan files its filesystem so that they can be moved between storage tiers, migrated, archived, etc. This is non-production work and has to be done in the background. When done with metadata on disk, the process becomes slower and slower as the number of files in a GPFS system rises and rises. So much so that, conceivably, eventually there aren't enough hours in the day to do it.

So IBM Research tried putting the metadata on flash arrays and seeing how much faster the system went. The result is impressive, very impressive, but not that surprising. It's also a tad, well, background, since the system wasn't handling real data.

Back in February, IBM announced a wondrous SONAS SPECsfs 2008 benchmark result of 403,326 SPECsfs ops/sec from a single GPFS system using 1,975 hard disk drives.

EMC bounded past this with a flash-heavy VNX system doing 497,623 ops/sec, using 436 x 200GB, SAS SSDs and 8 file systems.

SONAS is based on GPFS. From where El Reg sits, a re-run of the IBM SONAS SPECsfs2008 benchmark looks feasible, but this time using a few Violin Memory Arrays to hold the SONAS data and so get to the 500,000 SPECsfs2008 ops/sec area and beyond. We have asked both IBM and Violin about this but didn't expect to get anything looking like a "Yes, we're doing this" reply.

Much to our surprise we received this from Bruce Hillsberg, director of storage systems, IBM Research – Almaden: "You are correct: if we were to re-run the SPECsfs benchmark on a SONAS system with using the technology described in the press release, we would see a significant performance improvement."

Enticing, isn't it? ®

Combat fraud and increase customer satisfaction

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
BOFH: Oh DO tell us what you think. *CLICK*
$%%&amp Oh dear, we've been cut *CLICK* Well hello *CLICK* You're breaking up...
AMD's 'Seattle' 64-bit ARM server chips now sampling, set to launch in late 2014
But they won't appear in SeaMicro Fabric Compute Systems anytime soon
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.