Feeds

Intel and Yahoo! spawn open-source 'Tashi' cluster

Big Data goes back to school

Remote control for virtualized desktops

Backed by Yahoo!, HP, and Intel, the computer science mavens at Carnegie Mellon University have added a new compute cluster to the worldwide Open Cirrus test bed, a collection of clusters designed to explorer the frontiers of interwebs-scale distributed computing.

The test bed now spans ten clusters at locations across the globe.

Open Cirrus was launched in 2006 by Yahoo!, HP, and Intel to advance so-called cloud computing through open collaboration between academia, government, and, yes, the corporate world. Carnegie Mellon was already using an Open Cirrus cluster set up on its campus by Intel Labs Pittsburgh, and it was one of the first universities to make use of Yahoo!'s 4,000-processor M45 cluster, meant to promote research on Hadoop, the open source distributed number-crunching platform.

Juiced with CPU and cash donation from Intel and a power management and cooling systems contribution from APC, the new Carnegie Mellon cluster spans 159 servers and 1,165 processing cores, 2.4 terabytes of memory, and almost 900 terabytes of storage. Greg Ganger, a professor of electrical and computer engineering and director of Carnegie Mellon’s Parallel Data Lab, tells The Reg that the new cluster will run Tashi, an open source "infrastructure cloud" platform similar to Amazon's EC2.

Unlike Eucalyptus - another open source mirror of EC2 - Tashi is meant specifically for Big Data - i.e. the sort of number crunching handles by Hadoop, which the university will run atop its Tashi installation. "We do things Amazon wouldn't do," Ganger said, "things tailored specifically to something like Hadoop."

Hadoop is based on the Google File System (GFS) and MapReduce technologies developed at the Mountain View Chocolate Factory. In 2004, the Google published a pair of papers on these distributed computing technologies, and the platforms were reconstructed in open source by Doug Cutting, the man known previously for developing Lucene, the open-source retrieval library. He called his project Hadoop, after his son's stuffed elephant.

The Apache-based Tashi was co-developed by Carnegie Mellon and Intel, and it's currently running on that Intel Open Cirrus cluster on the university's campus. This cluster spans 25 nodes with 1000 cores and 450 disks providing more than 400 terabytes of storage.

With the new cluster, Ganger says his Parallel Data Labs will explorees all sorts of data intensive apps, including machine translation between languages like English and Chinese and deep analysis of the way netizens interact on the world's social networking sites. ®

Security for virtualized datacentres

More from The Register

next story
Just don't blame Bono! Apple iTunes music sales PLUMMET
Cupertino revenue hit by cheapo downloads, says report
The DRUGSTORES DON'T WORK, CVS makes IT WORSE ... for Apple Pay
Goog Wallet apparently also spurned in NFC lockdown
IBM, backing away from hardware? NEVER!
Don't be so sure, so-surers
Hey - who wants 4.8 TERABYTES almost AS FAST AS MEMORY?
China's Memblaze says they've got it in PCIe. Yow
Microsoft brings the CLOUD that GOES ON FOREVER
Sky's the limit with unrestricted space in the cloud
This time it's SO REAL: Overcoming the open-source orgasm myth with TODO
If the web giants need it to work, hey, maybe it'll work
'ANYTHING BUT STABLE' Netflix suffers BIG Europe-wide outage
Friday night LIVE? Nope. The only thing streaming are tears down my face
Google roolz! Nest buys Revolv, KILLS new sales of home hub
Take my temperature, I'm feeling a little bit dizzy
prev story

Whitepapers

Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
New hybrid storage solutions
Tackling data challenges through emerging hybrid storage solutions that enable optimum database performance whilst managing costs and increasingly large data stores.
Mitigating web security risk with SSL certificates
Web-based systems are essential tools for running business processes and delivering services to customers.