Feeds

'Grid computing Red Hat' out-Amazons Amazon

Cloudera in the, yes, cloud

Choosing a cloud hosting partner with confidence

Hadoop Summit In its mission to bring to world+dog the joys of Hadoop - that open-source grid-computing platform based on Google arrogance - Cloudera has out-Amazoned Amazon.

Today, the star-studded Hadoop startup told the world that its commercial stuffed-elephant distro can now be run on Amazon's Elastic Compute Cloud (EC2) in tandem with so-called Elastic Block Store (EBS) storage volumes. EBS volumes are mounted directly onto EC2 server instances.

This means you can run ongoing Hadoop jobs - starting them and stopping them whenever you like - without moving data back and forth between the local EC2 disks and Amazon's Simple Storage Sevice (S3). "Instead of using local disks, you can use EBS volumes," Cloudera man Christophe Bisciglia said today at the annual Hadoop Summit in Santa Clara, California.

"What's key about this is that your data is persistent. Currently, if you bring up a Hadoop cluster on Amazon and then bring it down, your [Hadoop File System] instance goes away. S3 can mitigate this, but then you have to round-trip between S3 and Hadoop every time you run a job.

"This is a way to turn your clusters on and off and keep them persistent and bring the full power of Hadoop."

Cloudera also says that its EBS integration improves Hadoop performance on the Amazon cloud by allowing more disks per server. EC2 provides a limited number of local disks for each instance.

Named for a yellow stuffed elephant, Hadoop mimics Google's MapReduce framework, mapping epic data-crunching tasks across a sea of machines - i.e. splitting them into tiny sub-tasks - before reducing the results into one master calculation. You can run it your own data centers - as Yahoo!, Facebook, and many others do - or you could run on Amazon's cloud. Or, for that matter, another infrastructure cloud.

Amazon's cloud offers its own Hadoop implementation as a service. It's called Amazon Elastic MapReduce. But it doesn't dovetail with EBS.

Bisciglia called Cloudera's EBS integration "a beta."

Cloudera also announced that its commercial distro - think of Cloudera as Hadoop's Red Hat - now includes the latest versions of Hive and Pig, two languages for coding atop Hadoop. The distro now includes Hive 0.3 and Pig 0.2. The distro is available at clouder.com/hadoop.

And the company has released beta packages of Hadoop version 0.20. "Twenty is going to be a really important release - it's going to include both sets of APIs, both the new and the old ones," Bisciglia said. ®

Intelligent flash storage arrays

More from The Register

next story
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Sway: Microsoft's new Office app doesn't have an Undo function
Content aggregation, meet the workplace ... oh
Sign off my IT project or I’ll PHONE your MUM
Honestly, it’s a piece of piss
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
Admins! Never mind POODLE, there're NEW OpenSSL bugs to splat
Four new patches for open-source crypto libraries
Torvalds CONFESSES: 'I'm pretty good at alienating devs'
Admits to 'a metric ****load' of mistakes during work with Linux collaborators
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.