Feeds

Yahoo!'s open source elephant loses its daddy

Hadoop founder departure 'unrelated' to MS pact

Top three mobile application threats

Yahoo! is losing the founder of Hadoop, that increasingly popular open source grid platform based on Google's proprietary software infrastructure.

On September 1, after three and a half years with Yahoo!, Doug Cutting will join Cloudera, the commercial Hadoop startup that launched earlier this year. As reported by the New York Times, Cutting announced his departure from Yahoo! this morning at a company meeting.

His announcement comes a little more than a week after Yahoo! agreed to replace its homegrown search tech with Microsoft Bing, a move that will eventually see the destruction of Yahoo!'s largest Hadoop app: the Yahoo! Search Webmap, which provides the company's search engine with a database of all known web pages. But in speaking with The Times, Cutting said his move has nothing to do with the Microhoo pact.

"This has been in the works for awhile and is unrelated," he said. "I am definitely not leaving in any sort of protest, and the thing I like least about this move is that it might be perceived that way."

Cutting did not respond to a request for comment. But Cloudera CEO Mike Olson confirmed that the startup was in talks with Cutting before the Microhoo deal. "Doug's stature in the community as the founder of the project has made him a pretty interesting candidate for us for some time," Olson tells The Reg. "My conversations with Doug preceded [the Microhoo deal] by some amount of time."

Inspired by Google research papers describing Mountain View’s proprietary software infrastructure, Hadoop is a means of crunching epic amounts of data across a network of distributed machines. Cutting first developed the platform for use with Nutch, his open source web crawler, naming it after his son's yellow stuffed elephant. But he was soon hired to help spearhead the project as a Yahoo! employee. The Google-battling web giant became the largest contributor to the Apache-hosted project, and by the beginning of last year, Hadoop had found its way onto the company's production systems, including Webmap.

The platform also underpins such web services as Facebook and Powerset, the semantic search engine that's now part of Microsoft's Bing. But Yahoo! remained the center of the community - at least until now.

In the wake of the Microhoo deal, Yahoo! reaffirmed its commitment to Hadoop, saying it would still power non-search technologies at the company. "Don't Panic!," wrote Hadoop development VP Eric Baldeschwieler. "We are as committed as ever to building a world class open source Cloud Computing infrastructure and Apache Hadoop remains our solution for batch computing. Hadoop is used to solve many, many internet scale problems beyond search at Yahoo."

And Cloudera's Mike Olson believes Yahoo! will remain a major player in the project. "[Cutting's move] helps us in our standing in the community," Olson tells us, "but I don't think Yahoo!'s role is diminished in any way."

But this does seem like a major shift in the project's center. And though Cutting and Cloudera say they were talking before the Microhoo deal, the Microhoo pact is hardly the sort of thing that would put a damper on his move.

In a canned statement, Yahoo! wished Cutting well. "We are very happy to have had Doug as part of our Yahoo! Hadoop team for the past three and a half years," the company said. "In that time we’ve worked together to make Apache Hadoop the most powerful and widely used open source software for handling large data sets and computing at Internet-scale. Moving forward, we wish Doug the best in his new endeavors. We are looking forward to continuing to lead on innovation and investment in Hadoop, as well as to collaborating with Doug and the growing Hadoop community."

As Yahoo! continues to lose its new-found mojo, Cloudera's stock is on the rise. Think of the Silicon Valley-based startup as the Red Hat of the Hadoop world. Cutting joins an all-star lineup of tech veterans, including former Googler Christophe Bisciglia, who Google famously dispatched to the University of Washington to teach a course on what the company likes to call Big Data.

For the past two years, Yahoo! has hosted the annual Hadoop Summitt near its home base in Sunnyvale. But on October 2, it's Cloudera's turn to MC an east coast incarnation: Hadoop World: NYC. You can bet that a certain new hire will be in attendance. ®

Maximizing your infrastructure through virtualization

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
Captain Kirk sets phaser to SLAUGHTER after trying new Facebook app
William Shatner less-than-impressed by Zuck's celebrity-only app
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
Cheer up, Nokia fans. It can start making mobes again in 18 months
The real winner of the Nokia sale is *drumroll* ... Nokia
Mozilla fixes CRITICAL security holes in Firefox, urges v31 upgrade
Misc memory hazards 'could be exploited' - and guess what, one's a Javascript vuln
EU dons gloves, pokes Google's deals with Android mobe makers
El Reg cops a squint at investigatory letters
Chrome browser has been DRAINING PC batteries for YEARS
Google is only now fixing ancient, energy-sapping bug
Google shows off new Chrome OS look
Athena springs full-grown from Chromium project's head
prev story

Whitepapers

Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.