Feeds

Cleversafe cuddles up to MapReduce, kicks HDFS out of bed

Hey Hadoop, we think your filesystem sucks

Top 5 reasons to deploy VMware with Tegile

Object storage specialist Cleversafe is after a piece of Big Data analytics action, and has wheeled in MapReduce to make it happen. In the same stroke it rejected HDFS as vulnerable and wasteful of storage capacity.

MapReduce is the analysis part of Hadoop, the flavour-of-the-month open-source data hoarding software. It processes information stored in Hadoop's Distributed Filesystem (HDFS), which employs a single metadata server and makes three copies of files to protect against drive failures and similar upsets.

Object storage systems such as Cleversafe's dsNet are designed to store masses of data on masses of connected nodes. It can self-heal by distributing metadata (and some kind of content hash) around the system of Slicestor nodes. Instead of adding HDFS capabilities alongside dsNet it's adding MapReduce to each Slicestor node.

Cleversafe says it stores only one copy of the MapReduce data instead of three, which is cheaper as capacities grow from terabytes to exabytes and on to petabytes. It isn't vulnerable to a catastrophic loss of data access caused by a metadata server failure because the metadata is distributed and protected - which is better than Hadoop according to Cleversafe.

There is no need for separate MapReduce servers and most Slicestor MapReduce apps will access data in their node rather than going out across the network so the system is more efficient.

Adding Cleversafe nodes adds capacity, both for raw data and metadata, and increases overall performance, the company claimed. The dsNet system can scale up into the petabyte range and beyond that into exabytes, it added.

Lockheed Martin is said to be working with Cleversafe to develop a suitable version of dsNet for US federal government agencies.

The initial MapReduce version of dsNet should be ready by the end of the year in a Cleversafe 3.0 software release. Support for third-party management tools - think Cloudera and HortonWorks - is expected to be added in the first half of 2013 in a 3.1 release. Conveniently, these builds may need Slicestore node hardware upgrades to cope with the added processing, memory and network loads. ®

Beginner's guide to SSL certificates

More from The Register

next story
Ellison: Sparc M7 is Oracle's most important silicon EVER
'Acceleration engines' key to performance, security, Larry says
Oracle SHELLSHOCKER - data titan lists unpatchables
Database kingpin lists 32 products that can't be patched (yet) as GNU fixes second vuln
Lenovo to finish $2.1bn IBM x86 server gobble in October
A lighter snack than expected – but what's a few $100m between friends, eh?
Ello? ello? ello?: Facebook challenger in DDoS KNOCKOUT
Gets back up again after half an hour though
Hey, what's a STORAGE company doing working on Internet-of-Cars?
Boo - it's not a terabyte car, it's just predictive maintenance and that
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.