Feeds

Intel and Yahoo! spawn open-source 'Tashi' cluster

Big Data goes back to school

Top three mobile application threats

Backed by Yahoo!, HP, and Intel, the computer science mavens at Carnegie Mellon University have added a new compute cluster to the worldwide Open Cirrus test bed, a collection of clusters designed to explorer the frontiers of interwebs-scale distributed computing.

The test bed now spans ten clusters at locations across the globe.

Open Cirrus was launched in 2006 by Yahoo!, HP, and Intel to advance so-called cloud computing through open collaboration between academia, government, and, yes, the corporate world. Carnegie Mellon was already using an Open Cirrus cluster set up on its campus by Intel Labs Pittsburgh, and it was one of the first universities to make use of Yahoo!'s 4,000-processor M45 cluster, meant to promote research on Hadoop, the open source distributed number-crunching platform.

Juiced with CPU and cash donation from Intel and a power management and cooling systems contribution from APC, the new Carnegie Mellon cluster spans 159 servers and 1,165 processing cores, 2.4 terabytes of memory, and almost 900 terabytes of storage. Greg Ganger, a professor of electrical and computer engineering and director of Carnegie Mellon’s Parallel Data Lab, tells The Reg that the new cluster will run Tashi, an open source "infrastructure cloud" platform similar to Amazon's EC2.

Unlike Eucalyptus - another open source mirror of EC2 - Tashi is meant specifically for Big Data - i.e. the sort of number crunching handles by Hadoop, which the university will run atop its Tashi installation. "We do things Amazon wouldn't do," Ganger said, "things tailored specifically to something like Hadoop."

Hadoop is based on the Google File System (GFS) and MapReduce technologies developed at the Mountain View Chocolate Factory. In 2004, the Google published a pair of papers on these distributed computing technologies, and the platforms were reconstructed in open source by Doug Cutting, the man known previously for developing Lucene, the open-source retrieval library. He called his project Hadoop, after his son's stuffed elephant.

The Apache-based Tashi was co-developed by Carnegie Mellon and Intel, and it's currently running on that Intel Open Cirrus cluster on the university's campus. This cluster spans 25 nodes with 1000 cores and 450 disks providing more than 400 terabytes of storage.

With the new cluster, Ganger says his Parallel Data Labs will explorees all sorts of data intensive apps, including machine translation between languages like English and Chinese and deep analysis of the way netizens interact on the world's social networking sites. ®

Combat fraud and increase customer satisfaction

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Kingston DataTraveler MicroDuo: Turn your phone into a 72GB beast
USB-usiness in the front, micro-USB party in the back
AMD's 'Seattle' 64-bit ARM server chips now sampling, set to launch in late 2014
But they won't appear in SeaMicro Fabric Compute Systems anytime soon
Brit boffins use TARDIS to re-route data flows through time and space
'Traffic Assignment and Retiming Dynamics with Inherent Stability' algo can save ISPs big bucks
Microsoft's Nadella: SQL Server 2014 means we're all about data
Adds new big data tools in quest for 'ambient intelligence'
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
SANS - Survey on application security programs
In this whitepaper learn about the state of application security programs and practices of 488 surveyed respondents, and discover how mature and effective these programs are.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.