Feeds

Dell floats Cloud Hadoop clusters

Stuffed elephant rides PowerEdge iron

Mobile application security vulnerability report

Slaves do all the Hadoop work

Here are the feeds and speeds on the iron underneath the Dell-Cloudera bundle:

Dell-Cloudera server configurations

Hadoop server configuration in the Dell-Cloudera bundle

Dell is using two-socket C2100 servers through the whole cluster, and recommends six-core Xeon E5645 processors running at 2.4GHz and with 12MB of cache for the master and edge nodes; these machines get six 600GB SAS drives. Dell recommends that slave nodes get four-core Xeon E5606 processors running at 2.13GHz and with only 8MB of cache, which is less compute and heat, while jacking up the machines to a dozen 1TB SATA drives spinning at 7200 RPM. The master and edge nodes have RAID data protection, while the slaves are just a bunch of disks hanging off an LSI2008 SATA controller.

This base Hadoop cluster can't do much, obviously, with only three slaves. But you can gang up the slave nodes into pods of 60 slave nodes, and then put a dozen of these together to create a Hadoop cluster that scales up to 720 slaves.

To give you a sense of scale, Yahoo! had 40,000 servers running Hadoop across dozens of clusters at the beginning of this year running various workloads, and expected to significantly increase that server count before the end of 2011. The largest Yahoo! Hadoop clusters have around 4,000 machines.

Under the deal that Dell and Cloudera have worked out, they are offering collaborative support contracts for the hardware or software, according to Albanese. Customers can call either Dell or Cloudera for their tech support questions, but obviously software questions will be routed to Cloudera and hardware issues will fall in Dell's lap. Dell's sales force is the only one peddling the combined solution, however.

Albanese declined to comment on whether other server makers had lined up to create similar Hadoop stacks on their servers. But Hewlett-Packard and Fujitsu would be obvious choices. It seems far more likely that Oracle will create its own Hadoop distribution before partnering with Cloudera or anyone else with a commercially supported Hadoop stack. Silicon Graphics, Cray, and Super Micro as well as a slew of whitebox vendors might be interested in creating their own Hadoop stacks, too. IBM already has its own BigInsights distribution and would probably not be interested unless customers started clamoring for it.

The Dell-Cloudera stack will start shipping later in August, according to Igoe. Pricing for that base configuration, including a year of support for the Cloudera stack, will run between $118,000 and $124,000. The higher price tag is for 24x7 tech support.

Dell has been doing Hadoop engagements, sometimes on the open source software and sometimes in conjunction with Cloudera, and early on companies in the financial services, Internet media and hosting, and retailing businesses have shown the most interest thus far, says Igoe. He offers some interesting observations.

First, brick and mortar retailers as well as online retailers are very keen on using Hadoop to improve their sales and better understand what their customers and prospects are doing (and not doing) when they visit their Web sites. Interestingly, retailers are looking at Hadoop as a means of aggregating log files across their myriad IT equipment behind the firewall as a means of cutting their teeth on Hadoop processing before they apply the technology to their retailing data.

Ditto for a number of hosting companies, who have homegrown and third-party system monitoring tools to keep track of what is going on in their infrastructure. They are looking at Hadoop as an alternative to these tools. Having mastered Hadoop on these internal tasks, companies will build out their clusters and aggregate their Web and customer data and do analytics on it. ®

Bridging the IT gap between rising business demands and ageing tools

More from The Register

next story
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
Seagate chances ARM with NAS boxes for the SOHO crowd
There's an Atom-powered offering, too
prev story

Whitepapers

Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.