Feeds

Cleversafe cuddles up to MapReduce, kicks HDFS out of bed

Hey Hadoop, we think your filesystem sucks

Secure remote control for conventional and virtual desktops

Object storage specialist Cleversafe is after a piece of Big Data analytics action, and has wheeled in MapReduce to make it happen. In the same stroke it rejected HDFS as vulnerable and wasteful of storage capacity.

MapReduce is the analysis part of Hadoop, the flavour-of-the-month open-source data hoarding software. It processes information stored in Hadoop's Distributed Filesystem (HDFS), which employs a single metadata server and makes three copies of files to protect against drive failures and similar upsets.

Object storage systems such as Cleversafe's dsNet are designed to store masses of data on masses of connected nodes. It can self-heal by distributing metadata (and some kind of content hash) around the system of Slicestor nodes. Instead of adding HDFS capabilities alongside dsNet it's adding MapReduce to each Slicestor node.

Cleversafe says it stores only one copy of the MapReduce data instead of three, which is cheaper as capacities grow from terabytes to exabytes and on to petabytes. It isn't vulnerable to a catastrophic loss of data access caused by a metadata server failure because the metadata is distributed and protected - which is better than Hadoop according to Cleversafe.

There is no need for separate MapReduce servers and most Slicestor MapReduce apps will access data in their node rather than going out across the network so the system is more efficient.

Adding Cleversafe nodes adds capacity, both for raw data and metadata, and increases overall performance, the company claimed. The dsNet system can scale up into the petabyte range and beyond that into exabytes, it added.

Lockheed Martin is said to be working with Cleversafe to develop a suitable version of dsNet for US federal government agencies.

The initial MapReduce version of dsNet should be ready by the end of the year in a Cleversafe 3.0 software release. Support for third-party management tools - think Cloudera and HortonWorks - is expected to be added in the first half of 2013 in a 3.1 release. Conveniently, these builds may need Slicestore node hardware upgrades to cope with the added processing, memory and network loads. ®

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
NSA SOURCE CODE LEAK: Information slurp tools to appear online
Now you can run your own intelligence agency
Azure TITSUP caused by INFINITE LOOP
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Stop the IoT revolution! We need to figure out packet sizes first
Researchers test 802.15.4 and find we know nuh-think! about large scale sensor network ops
SanDisk vows: We'll have a 16TB SSD WHOPPER by 2016
Flash WORM has a serious use for archived photos and videos
Astro-boffins start opening universe simulation data
Got a supercomputer? Want to simulate a universe? Here you go
prev story

Whitepapers

Go beyond APM with real-time IT operations analytics
How IT operations teams can harness the wealth of wire data already flowing through their environment for real-time operational intelligence.
5 critical considerations for enterprise cloud backup
Key considerations when evaluating cloud backup solutions to ensure adequate protection security and availability of enterprise data.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Simplify SSL certificate management across the enterprise
Simple steps to take control of SSL across the enterprise, and recommendations for a management platform for full visibility and single-point of control for these Certificates.