Feeds

Cloudera promises 'Google-like' Big Data dream in minutes

Hadoop shop automates so you don't have to

3 Big data security analytics techniques

Updated Cloudera has delivered a "substantial" update to its open source Hadoop distribution.

On Wednesday, Cloudera rolled out Cloudera Enterprise 3.5, two months after shipping a major upgrade to its Hadoop distribution called Cloudera Distribution of Apache Hadoop (CDH) 3.0.

Whereas CDH 3.0 expanded Cloudera's Hadoop stack from three components to 10, the idea behind Cloudera Enterprise 3.5 is to make that distro easier to manage and deploy for IT shops outside the ranks of Hadoop's super-users like Facebook, Yahoo!, and LinkedIn. Cloudera provides services and support for such mainstream users.

The changes will let you install and configure a full Google-like infrastructure "in a couple of minutes", product vice president Charles Zedlewski told The Register. "We have done a substantial update." Hadoop is based on Google's GFS and MapReduce platforms.

Zedlewski said Cloudera Enterprise 3.5 automates configuration changes, service restarts, and the addition and removal of hardware. There's also an Activity Monitor that consolidates user activity across components to provide both a real-time and historical view of user activities and jobs.

"[We have] expanded the capability of the management suite from monitoring and discovery of issues, to diagnostic problems, to automating changes, and setting long-term changes," Zedlewski said.

Cloudera has also enhanced the Hadoop Resource and Authorization Manager, facilitating rollbacks and improving security with LDAP systems.

Hadoop is an architecture for crunching huge amounts of data using a network of distributed servers. Nutch web crawler creator Doug Cutting based the platform on research papers describing Google GFS and MapReduce., and it is now an Apache Software Foundation (ASF) project.

Today, Cloudera also released a free "Express" edition of its Service and Configuration Manager module used in Cloudera Enterprise 3.5 that will automate the installation and configuration of Hadoop on a cluster of up to 50 nodes. Meanwhile, the company has also donated code for its packaging and testing suite to ASF, under a project called Bigtop. The idea is to help improve packaging and interoperability testing for Hadoop and related modules.

Among Bigtop's initial committers is Canonical, chief commercial steward of Ubuntu. Cloudera has supported packaging for Ubuntu Linux for the last two-and-a-half years.

Zedlewski said Cloudera will add a further three or four modules to the current stack, among them a compression algorithm that leverages Google's Snappy to speed up data import and export.

This will be added in an update to CDH in the next month or so, Zedlewski said. Other components are due in "the CDH 4 timeframe", he said, while Cloudera is also looking at enhancements around high-availability features in the core Hadoop module. Zedlewski would not provide a date for CDH 4, but he said "work is well underway". ®

This article has been updated to clarify Cloudera has delivered Cloudera Enterprise 3.5.

Combat fraud and increase customer satisfaction

More from The Register

next story
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Audio fans, prepare yourself for the Second Coming ... of Blu-ray
High Fidelity Pure Audio – is this what your ears have been waiting for?
Did a date calculation bug just cost hard-up Co-op Bank £110m?
And just when Brit banking org needs £400m to stay afloat
MtGox chief Karpelès refuses to come to US for g-men's grilling
Bitcoin baron says he needs another lawyer for FinCEN chat
Zucker punched: Google gobbles Facebook-wooed Titan Aerospace
Up, up and away in my beautiful balloon flying broadband-bot
Apple DOMINATES the Valley, rakes in more profit than Google, HP, Intel, Cisco COMBINED
Cook & Co. also pay more taxes than those four worthies PLUS eBay and Oracle
It may be ILLEGAL to run Heartbleed health checks – IT lawyer
Do the right thing, earn up to 10 years in clink
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.