Feeds

Yahoo!'s open source elephant loses its daddy

Hadoop founder departure 'unrelated' to MS pact

Choosing a cloud hosting partner with confidence

Yahoo! is losing the founder of Hadoop, that increasingly popular open source grid platform based on Google's proprietary software infrastructure.

On September 1, after three and a half years with Yahoo!, Doug Cutting will join Cloudera, the commercial Hadoop startup that launched earlier this year. As reported by the New York Times, Cutting announced his departure from Yahoo! this morning at a company meeting.

His announcement comes a little more than a week after Yahoo! agreed to replace its homegrown search tech with Microsoft Bing, a move that will eventually see the destruction of Yahoo!'s largest Hadoop app: the Yahoo! Search Webmap, which provides the company's search engine with a database of all known web pages. But in speaking with The Times, Cutting said his move has nothing to do with the Microhoo pact.

"This has been in the works for awhile and is unrelated," he said. "I am definitely not leaving in any sort of protest, and the thing I like least about this move is that it might be perceived that way."

Cutting did not respond to a request for comment. But Cloudera CEO Mike Olson confirmed that the startup was in talks with Cutting before the Microhoo deal. "Doug's stature in the community as the founder of the project has made him a pretty interesting candidate for us for some time," Olson tells The Reg. "My conversations with Doug preceded [the Microhoo deal] by some amount of time."

Inspired by Google research papers describing Mountain View’s proprietary software infrastructure, Hadoop is a means of crunching epic amounts of data across a network of distributed machines. Cutting first developed the platform for use with Nutch, his open source web crawler, naming it after his son's yellow stuffed elephant. But he was soon hired to help spearhead the project as a Yahoo! employee. The Google-battling web giant became the largest contributor to the Apache-hosted project, and by the beginning of last year, Hadoop had found its way onto the company's production systems, including Webmap.

The platform also underpins such web services as Facebook and Powerset, the semantic search engine that's now part of Microsoft's Bing. But Yahoo! remained the center of the community - at least until now.

In the wake of the Microhoo deal, Yahoo! reaffirmed its commitment to Hadoop, saying it would still power non-search technologies at the company. "Don't Panic!," wrote Hadoop development VP Eric Baldeschwieler. "We are as committed as ever to building a world class open source Cloud Computing infrastructure and Apache Hadoop remains our solution for batch computing. Hadoop is used to solve many, many internet scale problems beyond search at Yahoo."

And Cloudera's Mike Olson believes Yahoo! will remain a major player in the project. "[Cutting's move] helps us in our standing in the community," Olson tells us, "but I don't think Yahoo!'s role is diminished in any way."

But this does seem like a major shift in the project's center. And though Cutting and Cloudera say they were talking before the Microhoo deal, the Microhoo pact is hardly the sort of thing that would put a damper on his move.

In a canned statement, Yahoo! wished Cutting well. "We are very happy to have had Doug as part of our Yahoo! Hadoop team for the past three and a half years," the company said. "In that time we’ve worked together to make Apache Hadoop the most powerful and widely used open source software for handling large data sets and computing at Internet-scale. Moving forward, we wish Doug the best in his new endeavors. We are looking forward to continuing to lead on innovation and investment in Hadoop, as well as to collaborating with Doug and the growing Hadoop community."

As Yahoo! continues to lose its new-found mojo, Cloudera's stock is on the rise. Think of the Silicon Valley-based startup as the Red Hat of the Hadoop world. Cutting joins an all-star lineup of tech veterans, including former Googler Christophe Bisciglia, who Google famously dispatched to the University of Washington to teach a course on what the company likes to call Big Data.

For the past two years, Yahoo! has hosted the annual Hadoop Summitt near its home base in Sunnyvale. But on October 2, it's Cloudera's turn to MC an east coast incarnation: Hadoop World: NYC. You can bet that a certain new hire will be in attendance. ®

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
Nexus 7 fandroids tell of salty taste after sucking on Google's Lollipop
Web giant looking into why version 5.0 of Android is crippling older slabs
Bada-Bing! Mozilla flips Firefox to YAHOO! for search
Microsoft system will be the default for browser in US until 2020
Be real, Apple: In-app goodie grab games AREN'T FREE – EU
Cupertino stands down after Euro legal threats
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
SLURP! Flick your TONGUE around our LOLLIPOP – Google
Android 5 is coming – IF you're lucky enough to have the right gadget
Microsoft: Your Linux Docker containers are now OURS to command
New tool lets admins wrangle Linux apps from Windows
prev story

Whitepapers

Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Reducing the cost and complexity of web vulnerability management
How using vulnerability assessments to identify exploitable weaknesses and take corrective action can reduce the risk of hackers finding your site and attacking it.
Top 5 reasons to deploy VMware with Tegile
Data demand and the rise of virtualization is challenging IT teams to deliver storage performance, scalability and capacity that can keep up, while maximizing efficiency.