Feeds

OpenSFS to fund Lustre HPC file system development

If anyone forks the code, it will be Oracle

Internet Security Threat Report 2014

Over half of the top HPC centres in the world are using the Lustre file system, so the latest move to fund its development should make a lot of people happy.

And there will be a lot of brainpower and resources backing up the move: namely, supercomputer maker Cray, HPC storage vendor Data Direct Networks, and two nuke labs funded by the US Department of Energy - the Lawrence Livermore National Laboratory and Oak Ridge National Laboratory.

Cray, Data Direct Networks and the Lawrence Livermore and Oak Ridgehave labs have pooled resources to create Open Scalable File Systems. OpenSFS, a non-profit based in California, will fund the development of the Lustre parallel file system.

The Lustre file system is an open source project under the control of Oracle, if any open source project can be said to be under anyone's control. Of the top 100 HPC centers in the world, 60 of them are using the Lustre file system to feed data to their supers, according to Galen Shipment, technology integration group leader at Oak Ridge and a board member of OpenSFS.

The problem is not that Oracle is not continuing development on Lustre - the company put the finishing touches on Lustre 2.0 back in August - but rather that Oracle is not as interested in the HPC market as Sun Microsystems - and it has no intention of offering commercial-grade support for Lustre 2.0. Nevertheless, it is still offering support for the earlier Lustre 1.8 release. This is a big problem for HPC labs.

You might think that the big nuke labs already have the smartest people in the world working for them and that, if anything, it should be they who would offer tech support to the rest of the HPC community for products such as Lustre. I certainly thought that, based on the raw brainpower at these labs. Also, Lawrence Livermore was where a lot of the original Lustre file system work was funded out of DOE and put into production to put it through the paces. But Mark Seager, Lawrence Livermore's assistant department head for advanced technology - and an OpenSFS board member - said this is not what Lustre customers want.

While the hotshot HPC shops using Lustre are able to handle level one tech support fine, and can even wade in and offer level two support to offer bug fixes for the easy stuff, that's about as far as it goes.

"We need that third level of support backing us up for deep problems," explains Seager. "We do not have the manpower to do that deep support."

The need for deep and official support, and a product roadmap and development process that could accept the input and requirements from all the customers and shape future releases to address their needs, was why Peter Braam, a researcher at Carnegie Mellon University and founder of the Lustre project, created Cluster File Systems in 2001. Sun Microsystems bought CFS in 2007, and Oracle ate Sun in January.

The situation around Lustre was murky enough that Whamcloud, a startup founded with $10m in private equity funding and some of the top people involved with the development of Lustre, burst on the scene at the end of July to chase Lustre support contracts and do development on the parallel file system and submit code back into the Oracle-controlled Lustre code base.

Internet Security Threat Report 2014

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
IBM storage revenues sink: 'We are disappointed,' says CEO
Time to put the storage biz up for sale?
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
'Urika': Cray unveils new 1,500-core big data crunching monster
6TB of DRAM, 38TB of SSD flash and 120TB of disk storage
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
SDI wars: WTF is software defined infrastructure?
This time we play for ALL the marbles
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.