Feeds

Yahoo! Hadoop! brain! spin-off! doomed! to! fail!

The brains already left

SANS - Survey on application security programs

Open...and Shut The once dominant Yahoo is apparently keen to compete with one of today's hottest startups, Cloudera, to own the affections of data plumbers everywhere.

According to The Wall Street Journal, Yahoo is actively considering spinning out its Apache Hadoop engineering team into a startup, possibly backed by Benchmark Capital. The intent? Make a bunch of money from one of the industry's most important technologies, one used by a Who's Who of enterprises. The likely outcome? Utter failure.

Yes, Yahoo still generates more than $6 billion each year. Yes, Apache Hadoop is hot. And, yes, Yahoo CEO Carol Bartz has a foul, trash-talking mouth. But no, the three don't necessarily translate into a billion-dollar opportunity for a Yahoo-sponsored Apache Hadoop startup.

After all, while the intent of the spin-off would be to clear a path for the startup to operate independently of Yahoo, the reality is that Yahoo would remain a major shareholder. Yahoo has seen essentially flat revenues for four years and long ago lost its reputation for being a growth leader. Give the company a board seat and it's hard to foresee much beyond bureaucracy. If VMware's experience is anything to go by, Yahoo is likely to get as many as two board seats.

Indeed, VMware's parent company, EMC, also chairs the VMware board with its CEO, Joseph Tucci. Carol Bartz, Yahoo's CEO, is a great executive, but would this new startup want her leading its board? Doubtful, especially since it could seriously complicate the startup's possible exits and partners.

It's possible, of course, that things could be structured in such a way as to minimize Yahoo's influence on the company. With the right team in place, such a startup could go far, assuming it could find the right people. It's doubtful, however, that those "right people" currently reside at Yahoo, as I'll explain below.

Benchmark's Rob Bearden, former COO at both JBoss and SpringSource, revealed that Benchmark has been actively courting Yahoo about funding the startup. I could easily see Bearden gathering his network of hard-core Java developers and business executives to take on the challenge. I know Bearden, and think highly of his business acumen. He'd be a formidable competitor, despite Cloudera's headstart.

But there's reason to think he'd be too late in pulling together the cream of the Apache Hadoop crop. After all, Cloudera already employs some of the most critical Apache Hadoop developers, including Apache Hadoop co-founders Doug Cutting and Mike Cafarella. Plenty remain at Yahoo, but it feels like the heart and soul of Apache Hadoop is already at Cloudera…and eBay, Facebook, Apple, etc. Few of the brightest Apache Hadoop lights remain at Yahoo.

And why would they? Yahoo's been a tough place to work for a long time. As noted, the number of Apache Hadoop developers employed by Yahoo has shrunk steadily over time, and the character of those developers has also changed. The driven start-up guys left a long time ago. There were reasons to stay at Yahoo, but working in an exciting start-up environment for a company that was transforming the industry wasn't on the list. The people who stuck around were paid well and worked a big-company schedule. In other words, the Apache Hadoop developers who remain at Yahoo are precisely the wrong sort to bet on in startup land.

But they might be the right sort to help drive Yahoo's business. I spoke with a source familiar with Yahoo's business, and he reminded me that Yahoo relies on Hadoop for all portal content personalization: mail, news, spam filtering, etc. This is strategic to Yahoo's business, while core Hadoop development is not.

Bartz would be smart to shed developers so as to trim costs but then, she's already doing that: Apache Hadoop developers have been leaving Yahoo for years. Trying to get a startup spun-out and funded seems like a foolish way to accomplish this goal, with little real chance of disrupting the current momentum in the industry.

Cloudera's business has been booming, too, making it tough on any late entrant to the Apache Hadoop race. While somewhat true in any business, it's particularly the case for open-source businesses.

Open source tends to be winner-takes-all, as open source expert Dave Rosenberg has argued. The first company to own a particular project or category "wins" that category. Middleware? JBoss. Database? MySQL. Web content management? Acquia/Drupal. And so on.

In the case of Apache Hadoop, Cloudera has been winning customers for over two years. Yahoo's startup would first have to sort out the mechanics of this new company, launch, market its Apache Hadoop distribution, overcome the market's suspicions about its involvement with the new company, and actually start to close deals.

Oh, and then there's the problem of building a truly enterprise-ready distribution of Apache Hadoop. Yahoo's Apache Hadoop team is in the business of developing features for a single customer: Yahoo. Running an enterprise software company is hard work. You don't get to spend your time coding up the cool new features -- you need to write a roadmap, ship on time and invest enormous effort in chasing big elephants for your first customers, in order to convince others to follow.

No matter how mature Yahoo thinks its distribution is, it has never been run in production anywhere other than Yahoo. That's not a recipe for immediate, or even long-term success.

This isn't to say that Yahoo couldn't succeed, but if it is to suggest that there may be more efficient ways paths to success. For one, Yahoo could invest in Cloudera. No, this wouldn't give it the same level of ownership or control, but it would give the company a stake in the market leader for Apache Hadoop services. For another, Benchmark could simply hire away Yahoo's best engineers and start a new Apache Hadoop-related company without Yahoo's involvement. This would give Benchmark a cleaner ownership structure without sacrificing anything in terms of intellectual property. (Apache Hadoop is liberally licensed, so anyone can download and use it.)

In fact, the only option that strikes me as doomed to fail is Yahoo starting up a company to provide support and other services around Apache Hadoop. We already have a successful company doing that, one that Yahoo is highly unlikely to beat. ®

Matt Asay is senior vice president of business development at Strobe, a startup that offers an open source framework for building mobile apps. He was formerly chief operating officer of Ubuntu commercial operation Canonical. With more than a decade spent in open source, Asay served as Alfreso's general manager for the Americas and vice president of business development, and he helped put Novell on its open source track. Asay is an emeritus board member of the Open Source Initiative (OSI). His column, Open...and Shut, appears twice a week on The Register.

3 Big data security analytics techniques

More from The Register

next story
Next Windows obsolescence panic is 450 days from … NOW!
The clock is ticking louder for Windows Server 2003 R2 users
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
Red Hat to ship RHEL 7 release candidate with a taste of container tech
Grab 'near-final' version of next Enterprise Linux next week
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Inside the Hekaton: SQL Server 2014's database engine deconstructed
Nadella's database sqares the circle of cheap memory vs speed
prev story

Whitepapers

Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.