Rackspace to ride Hortonworks elephant into the clouds

Yahoo! should! buy! back! Hadoop! spinout!

Security for virtualized datacentres

Hadoop World Rackspace Hosting has spent the past two years helping craft the OpenStack cloud control freak and getting it running its public and private cloud services. And now, to get more of the IT wallet, Rackspace wants to peddle more services atop OpenStack, whether it is humming in your shop or in its own glass houses. And one of the first services that Rackspace wants to plunk on top of OpenStack with a price tag on it is, not surprisingly, the Hadoop big data muncher.

As a proponent of full open source cloud control software, it is not surprise at all that Rackspace has partnered up with Hortonworks rather than Cloudera, MapR Technologies, or Greenplum, all of which have some proprietary code in their Hadoop distributions.

To be fair, Greenplum, the big data arm of storage giant EMC, has just opened up its Chorus management tool for Hadoop and data warehouse clusters and the only proprietary Hadoop code it ships is a repackaged version of MapR's M5 Edition called Greenplum MR; you can get a supported version of the Apache Hadoop stack called Greenplum HD, which has no proprietary stuff in it.

Every one of the major Hadoop distributors can trace its roots back to Yahoo!, where the MapReduce computing methodology and distributed file system were cloned from ideas based on systems in use by Google several generations ago to run its search engine.

Cloudera has Doug Cutting, Hadoop's creator, on its payroll as well as many other heavy hitters from inside the former Hadoop factory at Yahoo! But Hortonworks is literally a spin out of the Yahoo! tech team that worked on Hadoop after the search engine company open sourced it under an Apache license in 2006 and has the most stringent view of Hadoop being open source, down to the last bit.

"This is our first foray into Hadoop and a good alignment for us," Huw Edwards, manager of corporate development at Rackspace, tells El Reg, adding that while the deal is by no means exclusive and Rackspace is keeping its options open, and the fact that OpenStack and the Hortonworks Data Platform are completely open source makes both Rackspace and its clients feel more comfortable about not getting vendor lock-in.

Of course, the irony is that if there is only one true open source and supported version of Hadoop, then that is a kind of lock in, too. You need two vendors living on a shoestring to really be free, and the odds are the market will create a couple alternatives.

The details are a bit sketchy right now, but the plan is for Rackspace to work with Hortonworks to streamline the operation of HDP 1.0, which just shipped in June, with the Rackspace Cloud and tuned to work well with OpenStack.

One of the issues the companies are working on is moving data from a set of Cloud Servers, the name for virtual servers on Rackspace's cloudy infrastructure, to its Cloud Files object storage. Rackspace will allow customers to puff up and suck down Hadoop clouds on demand so they don't have to invest in infrastructure over the long haul if their jobs don't demand it.

So, for instance, you might collect telemetry data in real-time and plunk it directly into the Cloud Files service, but only chew on it every couple of hours, days, or weeks depending on your business and the nature of the data. Either way, you only fire up the Cloud Servers to do the chewing when you need to, and when you are not chewing, you can turn off the capacity and not spend money.

This Hadoop public cloud service will be followed by a private cloud offering that will have Rackspace techies manage a private cloud built to the same specs inside your own data center, much as it does with OpenStack infrastructure clouds today.

The Hadoop public cloud service will be available in early 2013, Edwards says, with the private Hadoop cloud service coming sometime in late 2013. You can contact Rackspace now if you want to do a custom engagement before the Hadoop big data service is available, or to get in line for the beta program.

By the way, Rackspace is by no means a newbie to Hadoop. The company has a number of customers using its hosted servers to run Hadoop clusters on bare metal, but as Edwards points out, this requires a significant capital investment on the front end. Rackspace has customers using Cloudera, MapR, Greenplum, and Hortonworks distros on these hosted machines.

Now it will be coming up against Amazon Web Services' Elastic MapReduce service, which runs the Apache Hadoop distribution as a service as well as the MapR M5 Edition. It won't be long – okay, on second thought, maybe it will be long – before Oracle, IBM, Hewlett-Packard, and Dell all fire up Hadoop clusters on their clouds.

With a new CEO and ex-Googler Marissa Mayer at the helm and the company flush with $3.6bn, you have to wonder if Mayer is thinking that maybe letting Hortonworks go was a bad idea. If her former employer demonstrated anything, it was that constantly inventing, smashing, and reinventing server, storage, database, file systems, and other technologies is the only way to extract big profits out of the intertubes. ®

Providing a secure and efficient Helpdesk

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
IBM storage revenues sink: 'We are disappointed,' says CEO
Time to put the storage biz up for sale?
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
Symantec backs out of Backup Exec: Plans to can appliance in Jan
Will still provide support to existing customers
prev story


Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.