Feeds

Yahoo! Hadoop! brain! spin-off! doomed! to! fail!

The brains already left

Beginner's guide to SSL certificates

Open...and Shut The once dominant Yahoo is apparently keen to compete with one of today's hottest startups, Cloudera, to own the affections of data plumbers everywhere.

According to The Wall Street Journal, Yahoo is actively considering spinning out its Apache Hadoop engineering team into a startup, possibly backed by Benchmark Capital. The intent? Make a bunch of money from one of the industry's most important technologies, one used by a Who's Who of enterprises. The likely outcome? Utter failure.

Yes, Yahoo still generates more than $6 billion each year. Yes, Apache Hadoop is hot. And, yes, Yahoo CEO Carol Bartz has a foul, trash-talking mouth. But no, the three don't necessarily translate into a billion-dollar opportunity for a Yahoo-sponsored Apache Hadoop startup.

After all, while the intent of the spin-off would be to clear a path for the startup to operate independently of Yahoo, the reality is that Yahoo would remain a major shareholder. Yahoo has seen essentially flat revenues for four years and long ago lost its reputation for being a growth leader. Give the company a board seat and it's hard to foresee much beyond bureaucracy. If VMware's experience is anything to go by, Yahoo is likely to get as many as two board seats.

Indeed, VMware's parent company, EMC, also chairs the VMware board with its CEO, Joseph Tucci. Carol Bartz, Yahoo's CEO, is a great executive, but would this new startup want her leading its board? Doubtful, especially since it could seriously complicate the startup's possible exits and partners.

It's possible, of course, that things could be structured in such a way as to minimize Yahoo's influence on the company. With the right team in place, such a startup could go far, assuming it could find the right people. It's doubtful, however, that those "right people" currently reside at Yahoo, as I'll explain below.

Benchmark's Rob Bearden, former COO at both JBoss and SpringSource, revealed that Benchmark has been actively courting Yahoo about funding the startup. I could easily see Bearden gathering his network of hard-core Java developers and business executives to take on the challenge. I know Bearden, and think highly of his business acumen. He'd be a formidable competitor, despite Cloudera's headstart.

But there's reason to think he'd be too late in pulling together the cream of the Apache Hadoop crop. After all, Cloudera already employs some of the most critical Apache Hadoop developers, including Apache Hadoop co-founders Doug Cutting and Mike Cafarella. Plenty remain at Yahoo, but it feels like the heart and soul of Apache Hadoop is already at Cloudera…and eBay, Facebook, Apple, etc. Few of the brightest Apache Hadoop lights remain at Yahoo.

And why would they? Yahoo's been a tough place to work for a long time. As noted, the number of Apache Hadoop developers employed by Yahoo has shrunk steadily over time, and the character of those developers has also changed. The driven start-up guys left a long time ago. There were reasons to stay at Yahoo, but working in an exciting start-up environment for a company that was transforming the industry wasn't on the list. The people who stuck around were paid well and worked a big-company schedule. In other words, the Apache Hadoop developers who remain at Yahoo are precisely the wrong sort to bet on in startup land.

But they might be the right sort to help drive Yahoo's business. I spoke with a source familiar with Yahoo's business, and he reminded me that Yahoo relies on Hadoop for all portal content personalization: mail, news, spam filtering, etc. This is strategic to Yahoo's business, while core Hadoop development is not.

Bartz would be smart to shed developers so as to trim costs but then, she's already doing that: Apache Hadoop developers have been leaving Yahoo for years. Trying to get a startup spun-out and funded seems like a foolish way to accomplish this goal, with little real chance of disrupting the current momentum in the industry.

Cloudera's business has been booming, too, making it tough on any late entrant to the Apache Hadoop race. While somewhat true in any business, it's particularly the case for open-source businesses.

Open source tends to be winner-takes-all, as open source expert Dave Rosenberg has argued. The first company to own a particular project or category "wins" that category. Middleware? JBoss. Database? MySQL. Web content management? Acquia/Drupal. And so on.

In the case of Apache Hadoop, Cloudera has been winning customers for over two years. Yahoo's startup would first have to sort out the mechanics of this new company, launch, market its Apache Hadoop distribution, overcome the market's suspicions about its involvement with the new company, and actually start to close deals.

Oh, and then there's the problem of building a truly enterprise-ready distribution of Apache Hadoop. Yahoo's Apache Hadoop team is in the business of developing features for a single customer: Yahoo. Running an enterprise software company is hard work. You don't get to spend your time coding up the cool new features -- you need to write a roadmap, ship on time and invest enormous effort in chasing big elephants for your first customers, in order to convince others to follow.

No matter how mature Yahoo thinks its distribution is, it has never been run in production anywhere other than Yahoo. That's not a recipe for immediate, or even long-term success.

This isn't to say that Yahoo couldn't succeed, but if it is to suggest that there may be more efficient ways paths to success. For one, Yahoo could invest in Cloudera. No, this wouldn't give it the same level of ownership or control, but it would give the company a stake in the market leader for Apache Hadoop services. For another, Benchmark could simply hire away Yahoo's best engineers and start a new Apache Hadoop-related company without Yahoo's involvement. This would give Benchmark a cleaner ownership structure without sacrificing anything in terms of intellectual property. (Apache Hadoop is liberally licensed, so anyone can download and use it.)

In fact, the only option that strikes me as doomed to fail is Yahoo starting up a company to provide support and other services around Apache Hadoop. We already have a successful company doing that, one that Yahoo is highly unlikely to beat. ®

Matt Asay is senior vice president of business development at Strobe, a startup that offers an open source framework for building mobile apps. He was formerly chief operating officer of Ubuntu commercial operation Canonical. With more than a decade spent in open source, Asay served as Alfreso's general manager for the Americas and vice president of business development, and he helped put Novell on its open source track. Asay is an emeritus board member of the Open Source Initiative (OSI). His column, Open...and Shut, appears twice a week on The Register.

Remote control for virtualized desktops

More from The Register

next story
Nexus 7 fandroids tell of salty taste after sucking on Google's Lollipop
Web giant looking into why version 5.0 of Android is crippling older slabs
Be real, Apple: In-app goodie grab games AREN'T FREE – EU
Cupertino stands down after Euro legal threats
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
SLURP! Flick your TONGUE around our LOLLIPOP – Google
Android 5 is coming – IF you're lucky enough to have the right gadget
Microsoft: Your Linux Docker containers are now OURS to command
New tool lets admins wrangle Linux apps from Windows
Bada-Bing! Mozilla flips Firefox to YAHOO! for search
Microsoft system will be the default for browser in US until 2020
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
The hidden costs of self-signed SSL certificates
Exploring the true TCO for self-signed SSL certificates, including a side-by-side comparison of a self-signed architecture versus working with a third-party SSL vendor.
Top 5 reasons to deploy VMware with Tegile
Data demand and the rise of virtualization is challenging IT teams to deliver storage performance, scalability and capacity that can keep up, while maximizing efficiency.