Feeds

Big Blue boffins scan 10 billion files in Flash in a flash

Enticing developments at the lab

Remote control for virtualized desktops

IBM and Violin have announced a great big GPFS numbers record: the software scanned 10 billion files in a flash – well, 43 minutes – using four Violin flash memory arrays.

This was 37 times faster than a previous GPFS record of scanning one billion files in three hours, but that was with the file system metadata stored, like the file data, on disk drives.

Why does this matter? IBM says it is because GPFS needs to scan files its filesystem so that they can be moved between storage tiers, migrated, archived, etc. This is non-production work and has to be done in the background. When done with metadata on disk, the process becomes slower and slower as the number of files in a GPFS system rises and rises. So much so that, conceivably, eventually there aren't enough hours in the day to do it.

So IBM Research tried putting the metadata on flash arrays and seeing how much faster the system went. The result is impressive, very impressive, but not that surprising. It's also a tad, well, background, since the system wasn't handling real data.

Back in February, IBM announced a wondrous SONAS SPECsfs 2008 benchmark result of 403,326 SPECsfs ops/sec from a single GPFS system using 1,975 hard disk drives.

EMC bounded past this with a flash-heavy VNX system doing 497,623 ops/sec, using 436 x 200GB, SAS SSDs and 8 file systems.

SONAS is based on GPFS. From where El Reg sits, a re-run of the IBM SONAS SPECsfs2008 benchmark looks feasible, but this time using a few Violin Memory Arrays to hold the SONAS data and so get to the 500,000 SPECsfs2008 ops/sec area and beyond. We have asked both IBM and Violin about this but didn't expect to get anything looking like a "Yes, we're doing this" reply.

Much to our surprise we received this from Bruce Hillsberg, director of storage systems, IBM Research – Almaden: "You are correct: if we were to re-run the SPECsfs benchmark on a SONAS system with using the technology described in the press release, we would see a significant performance improvement."

Enticing, isn't it? ®

Choosing a cloud hosting partner with confidence

More from The Register

next story
729 teraflops, 71,000-core Super cost just US$5,500 to build
Cloud doubters, this isn't going to be your best day
Want to STUFF Facebook with blatant ADVERTISING? Fine! But you must PAY
Pony up or push off, Zuck tells social marketeers
Oi, Europe! Tell US feds to GTFO of our servers, say Microsoft and pals
By writing a really angry letter about how it's harming our cloud business, ta
SAVE ME, NASA system builder, from my DEAD WORKSTATION
Anal-retentive hardware nerd in paws-on workstation crisis
Microsoft adds video offering to Office 365. Oh NOES, you'll need Adobe Flash
Lovely presentations... but not on your Flash-hating mobe
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Simplify SSL certificate management across the enterprise
Simple steps to take control of SSL across the enterprise, and recommendations for a management platform for full visibility and single-point of control for these Certificates.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.