Feeds

'Grid computing Red Hat' out-Amazons Amazon

Cloudera in the, yes, cloud

Providing a secure and efficient Helpdesk

Hadoop Summit In its mission to bring to world+dog the joys of Hadoop - that open-source grid-computing platform based on Google arrogance - Cloudera has out-Amazoned Amazon.

Today, the star-studded Hadoop startup told the world that its commercial stuffed-elephant distro can now be run on Amazon's Elastic Compute Cloud (EC2) in tandem with so-called Elastic Block Store (EBS) storage volumes. EBS volumes are mounted directly onto EC2 server instances.

This means you can run ongoing Hadoop jobs - starting them and stopping them whenever you like - without moving data back and forth between the local EC2 disks and Amazon's Simple Storage Sevice (S3). "Instead of using local disks, you can use EBS volumes," Cloudera man Christophe Bisciglia said today at the annual Hadoop Summit in Santa Clara, California.

"What's key about this is that your data is persistent. Currently, if you bring up a Hadoop cluster on Amazon and then bring it down, your [Hadoop File System] instance goes away. S3 can mitigate this, but then you have to round-trip between S3 and Hadoop every time you run a job.

"This is a way to turn your clusters on and off and keep them persistent and bring the full power of Hadoop."

Cloudera also says that its EBS integration improves Hadoop performance on the Amazon cloud by allowing more disks per server. EC2 provides a limited number of local disks for each instance.

Named for a yellow stuffed elephant, Hadoop mimics Google's MapReduce framework, mapping epic data-crunching tasks across a sea of machines - i.e. splitting them into tiny sub-tasks - before reducing the results into one master calculation. You can run it your own data centers - as Yahoo!, Facebook, and many others do - or you could run on Amazon's cloud. Or, for that matter, another infrastructure cloud.

Amazon's cloud offers its own Hadoop implementation as a service. It's called Amazon Elastic MapReduce. But it doesn't dovetail with EBS.

Bisciglia called Cloudera's EBS integration "a beta."

Cloudera also announced that its commercial distro - think of Cloudera as Hadoop's Red Hat - now includes the latest versions of Hive and Pig, two languages for coding atop Hadoop. The distro now includes Hive 0.3 and Pig 0.2. The distro is available at clouder.com/hadoop.

And the company has released beta packages of Hadoop version 0.20. "Twenty is going to be a really important release - it's going to include both sets of APIs, both the new and the old ones," Bisciglia said. ®

Providing a secure and efficient Helpdesk

More from The Register

next story
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
Mathematica hits the Web
Wolfram embraces the cloud, promies private cloud cut of its number-cruncher
Mozilla shutters Labs, tells nobody it's been dead for five months
Staffer's blog reveals all as projects languish on GitHub
'People have forgotten just how late the first iPhone arrived ...'
Plus: 'Google's IDEALISM is an injudicious justification for inappropriate biz practices'
SUSE Linux owner Attachmate gobbled by Micro Focus for $2.3bn
Merger will lead to mainframe and COBOL powerhouse
iOS 8 Healthkit gets a bug SO Apple KILLS it. That's real healthcare!
Not fit for purpose on day of launch, says Cupertino
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.