Rackspace to ride Hortonworks elephant into the clouds

Yahoo! should! buy! back! Hadoop! spinout!

Boost IT visibility and business value

Hadoop World Rackspace Hosting has spent the past two years helping craft the OpenStack cloud control freak and getting it running its public and private cloud services. And now, to get more of the IT wallet, Rackspace wants to peddle more services atop OpenStack, whether it is humming in your shop or in its own glass houses. And one of the first services that Rackspace wants to plunk on top of OpenStack with a price tag on it is, not surprisingly, the Hadoop big data muncher.

As a proponent of full open source cloud control software, it is not surprise at all that Rackspace has partnered up with Hortonworks rather than Cloudera, MapR Technologies, or Greenplum, all of which have some proprietary code in their Hadoop distributions.

To be fair, Greenplum, the big data arm of storage giant EMC, has just opened up its Chorus management tool for Hadoop and data warehouse clusters and the only proprietary Hadoop code it ships is a repackaged version of MapR's M5 Edition called Greenplum MR; you can get a supported version of the Apache Hadoop stack called Greenplum HD, which has no proprietary stuff in it.

Every one of the major Hadoop distributors can trace its roots back to Yahoo!, where the MapReduce computing methodology and distributed file system were cloned from ideas based on systems in use by Google several generations ago to run its search engine.

Cloudera has Doug Cutting, Hadoop's creator, on its payroll as well as many other heavy hitters from inside the former Hadoop factory at Yahoo! But Hortonworks is literally a spin out of the Yahoo! tech team that worked on Hadoop after the search engine company open sourced it under an Apache license in 2006 and has the most stringent view of Hadoop being open source, down to the last bit.

"This is our first foray into Hadoop and a good alignment for us," Huw Edwards, manager of corporate development at Rackspace, tells El Reg, adding that while the deal is by no means exclusive and Rackspace is keeping its options open, and the fact that OpenStack and the Hortonworks Data Platform are completely open source makes both Rackspace and its clients feel more comfortable about not getting vendor lock-in.

Of course, the irony is that if there is only one true open source and supported version of Hadoop, then that is a kind of lock in, too. You need two vendors living on a shoestring to really be free, and the odds are the market will create a couple alternatives.

The details are a bit sketchy right now, but the plan is for Rackspace to work with Hortonworks to streamline the operation of HDP 1.0, which just shipped in June, with the Rackspace Cloud and tuned to work well with OpenStack.

One of the issues the companies are working on is moving data from a set of Cloud Servers, the name for virtual servers on Rackspace's cloudy infrastructure, to its Cloud Files object storage. Rackspace will allow customers to puff up and suck down Hadoop clouds on demand so they don't have to invest in infrastructure over the long haul if their jobs don't demand it.

So, for instance, you might collect telemetry data in real-time and plunk it directly into the Cloud Files service, but only chew on it every couple of hours, days, or weeks depending on your business and the nature of the data. Either way, you only fire up the Cloud Servers to do the chewing when you need to, and when you are not chewing, you can turn off the capacity and not spend money.

This Hadoop public cloud service will be followed by a private cloud offering that will have Rackspace techies manage a private cloud built to the same specs inside your own data center, much as it does with OpenStack infrastructure clouds today.

The Hadoop public cloud service will be available in early 2013, Edwards says, with the private Hadoop cloud service coming sometime in late 2013. You can contact Rackspace now if you want to do a custom engagement before the Hadoop big data service is available, or to get in line for the beta program.

By the way, Rackspace is by no means a newbie to Hadoop. The company has a number of customers using its hosted servers to run Hadoop clusters on bare metal, but as Edwards points out, this requires a significant capital investment on the front end. Rackspace has customers using Cloudera, MapR, Greenplum, and Hortonworks distros on these hosted machines.

Now it will be coming up against Amazon Web Services' Elastic MapReduce service, which runs the Apache Hadoop distribution as a service as well as the MapR M5 Edition. It won't be long – okay, on second thought, maybe it will be long – before Oracle, IBM, Hewlett-Packard, and Dell all fire up Hadoop clusters on their clouds.

With a new CEO and ex-Googler Marissa Mayer at the helm and the company flush with $3.6bn, you have to wonder if Mayer is thinking that maybe letting Hortonworks go was a bad idea. If her former employer demonstrated anything, it was that constantly inventing, smashing, and reinventing server, storage, database, file systems, and other technologies is the only way to extract big profits out of the intertubes. ®

The essential guide to IT transformation

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
US regulators OK sale of IBM's x86 server biz to Lenovo
Now all that remains is for gov't offices to ban the boxes
Flash could be CHEAPER than SAS DISK? Come off it, NetApp
Stats analysis reckons we'll hit that point in just three years
Oracle reveals 32-core, 10 BEEELLION-transistor SPARC M7
New chip scales to 1024 cores, 8192 threads 64 TB RAM, at speeds over 3.6GHz
Object storage bods Exablox: RAID is dead, baby. RAID is dead
Bring your own disks to its object appliances
Nimble's latest mutants GORGE themselves on unlucky forerunners
Crossing Sandy Bridges without stopping for breath
prev story


5 things you didn’t know about cloud backup
IT departments are embracing cloud backup, but there’s a lot you need to know before choosing a service provider. Learn all the critical things you need to know.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Build a business case: developing custom apps
Learn how to maximize the value of custom applications by accelerating and simplifying their development.
Rethinking backup and recovery in the modern data center
Combining intelligence, operational analytics, and automation to enable efficient, data-driven IT organizations using the HP ABR approach.
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.