Feeds

Intel and Yahoo! spawn open-source 'Tashi' cluster

Big Data goes back to school

Protecting against web application threats using SSL

Backed by Yahoo!, HP, and Intel, the computer science mavens at Carnegie Mellon University have added a new compute cluster to the worldwide Open Cirrus test bed, a collection of clusters designed to explorer the frontiers of interwebs-scale distributed computing.

The test bed now spans ten clusters at locations across the globe.

Open Cirrus was launched in 2006 by Yahoo!, HP, and Intel to advance so-called cloud computing through open collaboration between academia, government, and, yes, the corporate world. Carnegie Mellon was already using an Open Cirrus cluster set up on its campus by Intel Labs Pittsburgh, and it was one of the first universities to make use of Yahoo!'s 4,000-processor M45 cluster, meant to promote research on Hadoop, the open source distributed number-crunching platform.

Juiced with CPU and cash donation from Intel and a power management and cooling systems contribution from APC, the new Carnegie Mellon cluster spans 159 servers and 1,165 processing cores, 2.4 terabytes of memory, and almost 900 terabytes of storage. Greg Ganger, a professor of electrical and computer engineering and director of Carnegie Mellon’s Parallel Data Lab, tells The Reg that the new cluster will run Tashi, an open source "infrastructure cloud" platform similar to Amazon's EC2.

Unlike Eucalyptus - another open source mirror of EC2 - Tashi is meant specifically for Big Data - i.e. the sort of number crunching handles by Hadoop, which the university will run atop its Tashi installation. "We do things Amazon wouldn't do," Ganger said, "things tailored specifically to something like Hadoop."

Hadoop is based on the Google File System (GFS) and MapReduce technologies developed at the Mountain View Chocolate Factory. In 2004, the Google published a pair of papers on these distributed computing technologies, and the platforms were reconstructed in open source by Doug Cutting, the man known previously for developing Lucene, the open-source retrieval library. He called his project Hadoop, after his son's stuffed elephant.

The Apache-based Tashi was co-developed by Carnegie Mellon and Intel, and it's currently running on that Intel Open Cirrus cluster on the university's campus. This cluster spans 25 nodes with 1000 cores and 450 disks providing more than 400 terabytes of storage.

With the new cluster, Ganger says his Parallel Data Labs will explorees all sorts of data intensive apps, including machine translation between languages like English and Chinese and deep analysis of the way netizens interact on the world's social networking sites. ®

Choosing a cloud hosting partner with confidence

More from The Register

next story
Wanna keep your data for 1,000 YEARS? No? Hard luck, HDS wants you to anyway
Combine Blu-ray and M-DISC and you get this monster
Google+ GOING, GOING ... ? Newbie Gmailers no longer forced into mandatory ID slurp
Mountain View distances itself from lame 'network thingy'
US boffins demo 'twisted radio' mux
OAM takes wireless signals to 32 Gbps
Apple flops out 2FA for iCloud in bid to stop future nude selfie leaks
Millions of 4chan users howl with laughter as Cupertino slams stable door
'Kim Kardashian snaps naked selfies with a BLACKBERRY'. *Twitterati gasps*
More alleged private, nude celeb pics appear online
Students playing with impressive racks? Yes, it's cluster comp time
The most comprehensive coverage the world has ever seen. Ever
Run little spreadsheet, run! IBM's Watson is coming to gobble you up
Big Blue's big super's big appetite for big data in big clouds for big analytics
Seagate's triple-headed Cerberus could SAVE the DISK WORLD
... and possibly bring us even more HAMR time. Yay!
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.