Feeds

SGI inks reseller deal with Cloudera

Hadoop clusters stacked and racked, data not included

HP ProLiant Gen8: Integrated lifecycle automation

Silicon Graphics is chasing elephants – stuffed elephants, that is – and it has enlisted Cloudera as a partner in its big-data safari.

Technically speaking, Silicon Graphics doesn't have to launch a Hadoop cluster business so that companies can do the kind of analytics that search engine giants such as Google and Yahoo! do on the unstructured data that encompasses the Web and how we surf around on it, leaving our trails of cookies and Web logs. Some of the biggest names on the Internet – including Yahoo!, Amazon.com, and Facebook – have already deployed the Hadoop MapReduce tool and its associated Hadoop Distributed File System on its iron, and some big US government agencies have tapped SGI to build Hadoop clusters – sometimes in containerized data centers.

SGI says that it has done Hadoop installations that span as many as 40,000 server node, and has done individual clusters that weight in at 4,000 nodes. That is the upper limit of Hadoop cluster size at Yahoo! and elsewhere these days, and according to insiders at Yahoo!, the search and media company has grown from 25,000 nodes running Hadoop – which is used to crunch clickstreams, scan emails, and serve up customized content – to 42,000 nodes with over 200PB of data in 2010. Yahoo expects to hit around 60,000 or so Hadoop nodes by the end of this year.

Not everyone has this scale of Hadoopery, of course. But just like every company thought during the dot-com boom that they needed to have a retail operation on the Web, every company during today's big-data boom thinks that every little bit of information on their systems is somehow valuable and can be mined for money. So they are saving everything and trying to figure out how to chew on it with tools like Hadoop.

For SGI, Hadoop clusters have been largely bespoke work, but in June the company began offering prefabbed Hadoop clusters, marrying its Rackable lines of power-efficient, double-sided rack servers and its Altix ICE supercomputing rack servers with the open source distribution of Hadoop maintained by the Apache Software Foundation, which inherited the Hadoop stack after it was developed by Yahoo! a number of years ago.

But, as is not surprising to anyone, governments and enterprises want commercial support for open source software, and they want systems that are not bespoke, but instead configured and tuned to work well together and delivered as a finished system that is ready to be loaded up with data.

SGI Hadoop cluster

SGI Hadoop clusters, unstructured data not included

So last Friday, SGI inked a reseller agreement with Cloudera, which fancies itself the Red Hat of the Hadoop distribution, and which includes some of the key people from Yahoo! who created the clone of the analytics behind an earlier generation of Google's search engine – Hadoop and its related Hadoop Distributed File System clone the MapReduce technique created by Google, which most certainly was never open source.

Under the agreement with Cloudera, SGI will distribute the current CDH3 distribution from Cloudera, and offer Level 1 technical support for the software as well. Cloudera is providing Level 2 and Level 3 support for the CDH3 software for now, but over time, says Bill Mannel, vice president of product marketing at SGI, the company will build up more Hadoop expertise and eventually offer Level 2 support and might even offer the whole enchilada.

At the moment, SGI is cooking up reference configurations on its Rackable machines, and hopes to have these cooked up by the Hadoop World conference in three weeks and shipping by the end of November.

In the meantime, if you are building a Hadoop cluster, SGI can build one in its factory and ship it to you ready to chew all that unruly data.

This time around, SGI plans to focus mostly on peddling its energy-efficient Rackable machines for Hadoop clusters, with the Altix ICE clusters, which are designed for HPC supercomputing workloads, taking a backseat. The company is also offering InfiniBand networking as an alternative to Ethernet for lashing the server nodes together, but it is not yet clear what advantage this may have or how that might affect the price of a Hadoop cluster.

"We are going to show bang, bang for the buck, and power savings," says Mannel, referring to the competitive advantage SGI thinks it can bring to Hadoop workloads. ®

Reducing security risks from open source software

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.