Feeds

Cleversafe cuddles up to MapReduce, kicks HDFS out of bed

Hey Hadoop, we think your filesystem sucks

3 Big data security analytics techniques

Object storage specialist Cleversafe is after a piece of Big Data analytics action, and has wheeled in MapReduce to make it happen. In the same stroke it rejected HDFS as vulnerable and wasteful of storage capacity.

MapReduce is the analysis part of Hadoop, the flavour-of-the-month open-source data hoarding software. It processes information stored in Hadoop's Distributed Filesystem (HDFS), which employs a single metadata server and makes three copies of files to protect against drive failures and similar upsets.

Object storage systems such as Cleversafe's dsNet are designed to store masses of data on masses of connected nodes. It can self-heal by distributing metadata (and some kind of content hash) around the system of Slicestor nodes. Instead of adding HDFS capabilities alongside dsNet it's adding MapReduce to each Slicestor node.

Cleversafe says it stores only one copy of the MapReduce data instead of three, which is cheaper as capacities grow from terabytes to exabytes and on to petabytes. It isn't vulnerable to a catastrophic loss of data access caused by a metadata server failure because the metadata is distributed and protected - which is better than Hadoop according to Cleversafe.

There is no need for separate MapReduce servers and most Slicestor MapReduce apps will access data in their node rather than going out across the network so the system is more efficient.

Adding Cleversafe nodes adds capacity, both for raw data and metadata, and increases overall performance, the company claimed. The dsNet system can scale up into the petabyte range and beyond that into exabytes, it added.

Lockheed Martin is said to be working with Cleversafe to develop a suitable version of dsNet for US federal government agencies.

The initial MapReduce version of dsNet should be ready by the end of the year in a Cleversafe 3.0 software release. Support for third-party management tools - think Cloudera and HortonWorks - is expected to be added in the first half of 2013 in a 3.1 release. Conveniently, these builds may need Slicestore node hardware upgrades to cope with the added processing, memory and network loads. ®

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
BOFH: Oh DO tell us what you think. *CLICK*
$%%&amp Oh dear, we've been cut *CLICK* Well hello *CLICK* You're breaking up...
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.