Feeds

Dell floats Cloud Hadoop clusters

Stuffed elephant rides PowerEdge iron

Next gen security for virtualised datacentres

Slaves do all the Hadoop work

Here are the feeds and speeds on the iron underneath the Dell-Cloudera bundle:

Dell-Cloudera server configurations

Hadoop server configuration in the Dell-Cloudera bundle

Dell is using two-socket C2100 servers through the whole cluster, and recommends six-core Xeon E5645 processors running at 2.4GHz and with 12MB of cache for the master and edge nodes; these machines get six 600GB SAS drives. Dell recommends that slave nodes get four-core Xeon E5606 processors running at 2.13GHz and with only 8MB of cache, which is less compute and heat, while jacking up the machines to a dozen 1TB SATA drives spinning at 7200 RPM. The master and edge nodes have RAID data protection, while the slaves are just a bunch of disks hanging off an LSI2008 SATA controller.

This base Hadoop cluster can't do much, obviously, with only three slaves. But you can gang up the slave nodes into pods of 60 slave nodes, and then put a dozen of these together to create a Hadoop cluster that scales up to 720 slaves.

To give you a sense of scale, Yahoo! had 40,000 servers running Hadoop across dozens of clusters at the beginning of this year running various workloads, and expected to significantly increase that server count before the end of 2011. The largest Yahoo! Hadoop clusters have around 4,000 machines.

Under the deal that Dell and Cloudera have worked out, they are offering collaborative support contracts for the hardware or software, according to Albanese. Customers can call either Dell or Cloudera for their tech support questions, but obviously software questions will be routed to Cloudera and hardware issues will fall in Dell's lap. Dell's sales force is the only one peddling the combined solution, however.

Albanese declined to comment on whether other server makers had lined up to create similar Hadoop stacks on their servers. But Hewlett-Packard and Fujitsu would be obvious choices. It seems far more likely that Oracle will create its own Hadoop distribution before partnering with Cloudera or anyone else with a commercially supported Hadoop stack. Silicon Graphics, Cray, and Super Micro as well as a slew of whitebox vendors might be interested in creating their own Hadoop stacks, too. IBM already has its own BigInsights distribution and would probably not be interested unless customers started clamoring for it.

The Dell-Cloudera stack will start shipping later in August, according to Igoe. Pricing for that base configuration, including a year of support for the Cloudera stack, will run between $118,000 and $124,000. The higher price tag is for 24x7 tech support.

Dell has been doing Hadoop engagements, sometimes on the open source software and sometimes in conjunction with Cloudera, and early on companies in the financial services, Internet media and hosting, and retailing businesses have shown the most interest thus far, says Igoe. He offers some interesting observations.

First, brick and mortar retailers as well as online retailers are very keen on using Hadoop to improve their sales and better understand what their customers and prospects are doing (and not doing) when they visit their Web sites. Interestingly, retailers are looking at Hadoop as a means of aggregating log files across their myriad IT equipment behind the firewall as a means of cutting their teeth on Hadoop processing before they apply the technology to their retailing data.

Ditto for a number of hosting companies, who have homegrown and third-party system monitoring tools to keep track of what is going on in their infrastructure. They are looking at Hadoop as an alternative to these tools. Having mastered Hadoop on these internal tasks, companies will build out their clusters and aggregate their Web and customer data and do analytics on it. ®

Gartner critical capabilities for enterprise endpoint backup

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
Cutting cancer rates: Data, models and a happy ending?
How surgery might be making cancer prognoses worse
Silicon Valley jolted by magnitude 6.1 quake – its biggest in 25 years
Did the earth move for you at VMworld – oh, OK. It just did. A lot
Forrester says it's time to give up on physical storage arrays
The physical/virtual storage tipping point may just have arrived
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
5 things you didn’t know about cloud backup
IT departments are embracing cloud backup, but there’s a lot you need to know before choosing a service provider. Learn all the critical things you need to know.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?