Feeds

Dell floats Cloud Hadoop clusters

Stuffed elephant rides PowerEdge iron

Internet Security Threat Report 2014

Slaves do all the Hadoop work

Here are the feeds and speeds on the iron underneath the Dell-Cloudera bundle:

Dell-Cloudera server configurations

Hadoop server configuration in the Dell-Cloudera bundle

Dell is using two-socket C2100 servers through the whole cluster, and recommends six-core Xeon E5645 processors running at 2.4GHz and with 12MB of cache for the master and edge nodes; these machines get six 600GB SAS drives. Dell recommends that slave nodes get four-core Xeon E5606 processors running at 2.13GHz and with only 8MB of cache, which is less compute and heat, while jacking up the machines to a dozen 1TB SATA drives spinning at 7200 RPM. The master and edge nodes have RAID data protection, while the slaves are just a bunch of disks hanging off an LSI2008 SATA controller.

This base Hadoop cluster can't do much, obviously, with only three slaves. But you can gang up the slave nodes into pods of 60 slave nodes, and then put a dozen of these together to create a Hadoop cluster that scales up to 720 slaves.

To give you a sense of scale, Yahoo! had 40,000 servers running Hadoop across dozens of clusters at the beginning of this year running various workloads, and expected to significantly increase that server count before the end of 2011. The largest Yahoo! Hadoop clusters have around 4,000 machines.

Under the deal that Dell and Cloudera have worked out, they are offering collaborative support contracts for the hardware or software, according to Albanese. Customers can call either Dell or Cloudera for their tech support questions, but obviously software questions will be routed to Cloudera and hardware issues will fall in Dell's lap. Dell's sales force is the only one peddling the combined solution, however.

Albanese declined to comment on whether other server makers had lined up to create similar Hadoop stacks on their servers. But Hewlett-Packard and Fujitsu would be obvious choices. It seems far more likely that Oracle will create its own Hadoop distribution before partnering with Cloudera or anyone else with a commercially supported Hadoop stack. Silicon Graphics, Cray, and Super Micro as well as a slew of whitebox vendors might be interested in creating their own Hadoop stacks, too. IBM already has its own BigInsights distribution and would probably not be interested unless customers started clamoring for it.

The Dell-Cloudera stack will start shipping later in August, according to Igoe. Pricing for that base configuration, including a year of support for the Cloudera stack, will run between $118,000 and $124,000. The higher price tag is for 24x7 tech support.

Dell has been doing Hadoop engagements, sometimes on the open source software and sometimes in conjunction with Cloudera, and early on companies in the financial services, Internet media and hosting, and retailing businesses have shown the most interest thus far, says Igoe. He offers some interesting observations.

First, brick and mortar retailers as well as online retailers are very keen on using Hadoop to improve their sales and better understand what their customers and prospects are doing (and not doing) when they visit their Web sites. Interestingly, retailers are looking at Hadoop as a means of aggregating log files across their myriad IT equipment behind the firewall as a means of cutting their teeth on Hadoop processing before they apply the technology to their retailing data.

Ditto for a number of hosting companies, who have homegrown and third-party system monitoring tools to keep track of what is going on in their infrastructure. They are looking at Hadoop as an alternative to these tools. Having mastered Hadoop on these internal tasks, companies will build out their clusters and aggregate their Web and customer data and do analytics on it. ®

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
Just don't blame Bono! Apple iTunes music sales PLUMMET
Cupertino revenue hit by cheapo downloads, says report
The DRUGSTORES DON'T WORK, CVS makes IT WORSE ... for Apple Pay
Goog Wallet apparently also spurned in NFC lockdown
IBM, backing away from hardware? NEVER!
Don't be so sure, so-surers
Hey - who wants 4.8 TERABYTES almost AS FAST AS MEMORY?
China's Memblaze says they've got it in PCIe. Yow
Microsoft brings the CLOUD that GOES ON FOREVER
Sky's the limit with unrestricted space in the cloud
This time it's SO REAL: Overcoming the open-source orgasm myth with TODO
If the web giants need it to work, hey, maybe it'll work
'ANYTHING BUT STABLE' Netflix suffers BIG Europe-wide outage
Friday night LIVE? Nope. The only thing streaming are tears down my face
Google roolz! Nest buys Revolv, KILLS new sales of home hub
Take my temperature, I'm feeling a little bit dizzy
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
New hybrid storage solutions
Tackling data challenges through emerging hybrid storage solutions that enable optimum database performance whilst managing costs and increasingly large data stores.
Protecting users from Firesheep and other Sidejacking attacks with SSL
Discussing the vulnerabilities inherent in Wi-Fi networks, and how using TLS/SSL for your entire site will assure security.