Feeds

Cleversafe cuddles up to MapReduce, kicks HDFS out of bed

Hey Hadoop, we think your filesystem sucks

High performance access to file storage

Object storage specialist Cleversafe is after a piece of Big Data analytics action, and has wheeled in MapReduce to make it happen. In the same stroke it rejected HDFS as vulnerable and wasteful of storage capacity.

MapReduce is the analysis part of Hadoop, the flavour-of-the-month open-source data hoarding software. It processes information stored in Hadoop's Distributed Filesystem (HDFS), which employs a single metadata server and makes three copies of files to protect against drive failures and similar upsets.

Object storage systems such as Cleversafe's dsNet are designed to store masses of data on masses of connected nodes. It can self-heal by distributing metadata (and some kind of content hash) around the system of Slicestor nodes. Instead of adding HDFS capabilities alongside dsNet it's adding MapReduce to each Slicestor node.

Cleversafe says it stores only one copy of the MapReduce data instead of three, which is cheaper as capacities grow from terabytes to exabytes and on to petabytes. It isn't vulnerable to a catastrophic loss of data access caused by a metadata server failure because the metadata is distributed and protected - which is better than Hadoop according to Cleversafe.

There is no need for separate MapReduce servers and most Slicestor MapReduce apps will access data in their node rather than going out across the network so the system is more efficient.

Adding Cleversafe nodes adds capacity, both for raw data and metadata, and increases overall performance, the company claimed. The dsNet system can scale up into the petabyte range and beyond that into exabytes, it added.

Lockheed Martin is said to be working with Cleversafe to develop a suitable version of dsNet for US federal government agencies.

The initial MapReduce version of dsNet should be ready by the end of the year in a Cleversafe 3.0 software release. Support for third-party management tools - think Cloudera and HortonWorks - is expected to be added in the first half of 2013 in a 3.1 release. Conveniently, these builds may need Slicestore node hardware upgrades to cope with the added processing, memory and network loads. ®

High performance access to file storage

More from The Register

next story
Seagate brings out 6TB HDD, did not need NO STEENKIN' SHINGLES
Or helium filling either, according to reports
European Court of Justice rips up Data Retention Directive
Rules 'interfering' measure to be 'invalid'
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
Bored with trading oil and gold? Why not flog some CLOUD servers?
Chicago Mercantile Exchange plans cloud spot exchange
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
prev story

Whitepapers

Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.