Feeds

Got no idea what Hadoop is, but think you need it? You're not alone

It won't happen without Appening

3 Big data security analytics techniques

Open ... and Shut Hadoop is quickly becoming essential infrastructure for enterprises hoping to glean insights from the massive quantities of data they collect. The problem is that relatively few enterprises have the necessary competence to make effective use of the still-complex open-source project. While Hadoop vendors like Cloudera, Hortonworks, EMC, and MapR are doing their parts to simplify Hadoop, the real breakthrough for Hadoop may come from the applications that run on it, and not improvements to the infrastructure, according to Cloudera CEO Mike Olson.

Olson joined Hadoop heavyweights from Metamarkets, Oracle, Facebook, and MapR to discuss "The Elephant in the Enterprise: What Role Will Hadoop Play?" at a Churchill Club event earlier this week. As the panel debated the pros and cons of Hadoop, it became clear that while Hadoop is the heartbeat of the Big Data movement today, it still has a long way to go to become mainstream.

From left: Metamarkets CEO Michael Driscoll; Andrew Mendelsohn, SVP, Oracle Server Technologies; Cloudera CEO Mike Olson; Jay Parikh, VP Infrastructure Engineering, Facebook; and MapR CEO John Schroeder at the panel discussion.

For example, Metamarkets CEO Michael Driscoll criticised Hadoop's dated batch processing in a world that increasingly demands real-time. "Hadoop is not like having a conversation with your data," he insisted. "Instead it's like having a pen pal that you write from time to time." To this Olson quipped, "To be fair to Hadoop, it's like having thousands of pen pals that write to you."

Apparently this is enough for now, with Hadoop skills topping the list of skills needed across the IT landscape, according to research firm Ovum and jobs data compiled by Indeed.com:

But how much of this is real versus media-inspired mania for the next big thing?

While Driscoll feels the need for Hadoop is real, he was also quick to caution that "we're in a Hadoop hangover" that has executives saying "I have no idea what Hadoop is but I think we need it." The problem, however, is that the reason so many executives want Hadoop (data analytics) is not what an enterprise gets out of the box. As Driscoll pointed out, "Hadoop is a technology, not a solution."

Which is why Olson is probably right when he declared, "There will be enormous Hadoop adoption, but you'll get it by virtue of the applications you run." Cloudera board member and Workday co-CEO Aneel Bhusri echoed this Olson sentiment:

In other words, Hadoop, like Linux, MySQL, PHP, MongoDB, and other open-source technology, is very important, but mostly because it will enable a rich ecosystem of applications running on top of it. Enterprises won't need to become Hadoop experts. The experts will be the SaaS providers who build Big Data applications that take advantage of Hadoop, but don't shove it in the user's face.

Driscoll's Metamarkets is set up on the premise that customers can tap into analytics as a service, with Hadoop running behind the scenes. Many enterprises, however, simply aren't yet comfortable pushing their data to the cloud, leading Oracle's SVP of Server Technologies, Andy Mendelsohn, to conclude that Hadoop appliances are a necessary intermediate step for cautious enterprises on the road to true Hadoop-based analytics-as-a-service.

In sum, Hadoop is great technology, but not yet a tool that mere mortals can manage. While Hadoop training will undoubtedly mint many more data scientists, the real value this Big Data technology promises to deliver is as a behind-the-scenes infrastructure for cloud services, including everything from Facebook to Workday to Metamarkets. While the panel participants didn't estimate the timing for an onslaught of new Big Data applications, there's enough money at stake that the answer is almost certainly: soon. Very soon. ®

Matt Asay is senior vice president of business development at Nodeable, offering systems management for managing and analysing cloud-based data. He was formerly SVP of biz dev at HTML5 start-up Strobe and chief operating officer of Ubuntu commercial operation Canonical. With more than a decade spent in open source, Asay served as Alfresco's general manager for the Americas and vice president of business development, and he helped put Novell on its open source track. Asay is an emeritus board member of the Open Source Initiative (OSI). His column, Open...and Shut, appears three times a week on The Register.

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Bored with trading oil and gold? Why not flog some CLOUD servers?
Chicago Mercantile Exchange plans cloud spot exchange
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
Amazon reveals its Google-killing 'R3' server instances
A mega-memory instance that never forgets
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.