Feeds

Rackspace to ride Hortonworks elephant into the clouds

Yahoo! should! buy! back! Hadoop! spinout!

The essential guide to IT transformation

Hadoop World Rackspace Hosting has spent the past two years helping craft the OpenStack cloud control freak and getting it running its public and private cloud services. And now, to get more of the IT wallet, Rackspace wants to peddle more services atop OpenStack, whether it is humming in your shop or in its own glass houses. And one of the first services that Rackspace wants to plunk on top of OpenStack with a price tag on it is, not surprisingly, the Hadoop big data muncher.

As a proponent of full open source cloud control software, it is not surprise at all that Rackspace has partnered up with Hortonworks rather than Cloudera, MapR Technologies, or Greenplum, all of which have some proprietary code in their Hadoop distributions.

To be fair, Greenplum, the big data arm of storage giant EMC, has just opened up its Chorus management tool for Hadoop and data warehouse clusters and the only proprietary Hadoop code it ships is a repackaged version of MapR's M5 Edition called Greenplum MR; you can get a supported version of the Apache Hadoop stack called Greenplum HD, which has no proprietary stuff in it.

Every one of the major Hadoop distributors can trace its roots back to Yahoo!, where the MapReduce computing methodology and distributed file system were cloned from ideas based on systems in use by Google several generations ago to run its search engine.

Cloudera has Doug Cutting, Hadoop's creator, on its payroll as well as many other heavy hitters from inside the former Hadoop factory at Yahoo! But Hortonworks is literally a spin out of the Yahoo! tech team that worked on Hadoop after the search engine company open sourced it under an Apache license in 2006 and has the most stringent view of Hadoop being open source, down to the last bit.

"This is our first foray into Hadoop and a good alignment for us," Huw Edwards, manager of corporate development at Rackspace, tells El Reg, adding that while the deal is by no means exclusive and Rackspace is keeping its options open, and the fact that OpenStack and the Hortonworks Data Platform are completely open source makes both Rackspace and its clients feel more comfortable about not getting vendor lock-in.

Of course, the irony is that if there is only one true open source and supported version of Hadoop, then that is a kind of lock in, too. You need two vendors living on a shoestring to really be free, and the odds are the market will create a couple alternatives.

The details are a bit sketchy right now, but the plan is for Rackspace to work with Hortonworks to streamline the operation of HDP 1.0, which just shipped in June, with the Rackspace Cloud and tuned to work well with OpenStack.

One of the issues the companies are working on is moving data from a set of Cloud Servers, the name for virtual servers on Rackspace's cloudy infrastructure, to its Cloud Files object storage. Rackspace will allow customers to puff up and suck down Hadoop clouds on demand so they don't have to invest in infrastructure over the long haul if their jobs don't demand it.

So, for instance, you might collect telemetry data in real-time and plunk it directly into the Cloud Files service, but only chew on it every couple of hours, days, or weeks depending on your business and the nature of the data. Either way, you only fire up the Cloud Servers to do the chewing when you need to, and when you are not chewing, you can turn off the capacity and not spend money.

This Hadoop public cloud service will be followed by a private cloud offering that will have Rackspace techies manage a private cloud built to the same specs inside your own data center, much as it does with OpenStack infrastructure clouds today.

The Hadoop public cloud service will be available in early 2013, Edwards says, with the private Hadoop cloud service coming sometime in late 2013. You can contact Rackspace now if you want to do a custom engagement before the Hadoop big data service is available, or to get in line for the beta program.

By the way, Rackspace is by no means a newbie to Hadoop. The company has a number of customers using its hosted servers to run Hadoop clusters on bare metal, but as Edwards points out, this requires a significant capital investment on the front end. Rackspace has customers using Cloudera, MapR, Greenplum, and Hortonworks distros on these hosted machines.

Now it will be coming up against Amazon Web Services' Elastic MapReduce service, which runs the Apache Hadoop distribution as a service as well as the MapR M5 Edition. It won't be long – okay, on second thought, maybe it will be long – before Oracle, IBM, Hewlett-Packard, and Dell all fire up Hadoop clusters on their clouds.

With a new CEO and ex-Googler Marissa Mayer at the helm and the company flush with $3.6bn, you have to wonder if Mayer is thinking that maybe letting Hortonworks go was a bad idea. If her former employer demonstrated anything, it was that constantly inventing, smashing, and reinventing server, storage, database, file systems, and other technologies is the only way to extract big profits out of the intertubes. ®

Boost IT visibility and business value

More from The Register

next story
Pay to play: The hidden cost of software defined everything
Enter credit card details if you want that system you bought to actually be useful
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
HP busts out new ProLiant Gen9 servers
Think those are cool? Wait till you get a load of our racks
Silicon Valley jolted by magnitude 6.1 quake – its biggest in 25 years
Did the earth move for you at VMworld – oh, OK. It just did. A lot
VMware's high-wire balancing act: EVO might drag us ALL down
Get it right, EMC, or there'll be STORAGE CIVIL WAR. Mark my words
Forrester says it's time to give up on physical storage arrays
The physical/virtual storage tipping point may just have arrived
prev story

Whitepapers

Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Backing up distributed data
Eliminating the redundant use of bandwidth and storage capacity and application consolidation in the modern data center.
The essential guide to IT transformation
ServiceNow discusses three IT transformations that can help CIOs automate IT services to transform IT and the enterprise
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.