Feeds

Yahoofrastructure swells in face of Microsoft pact

Yes, we're a technology company

Reducing the cost and complexity of web vulnerability management

Though it will eventually offload search to Steve Ballmer and Microsoft, Yahoo! has no intention of shrinking the new back-end infrastructure now driving its search technologies.

"What impact does the Microsoft search deal have on the Yahoo! cloud? It has no impact," Shelton Shugar, Yahoo!'s senior vp of cloud computing, tells The Reg. "All the services we're building, we will continue to build. All the roadmaps we have in place, we will continue to work towards."

For the past few years, Yahoo! has worked to build a unified collection of back-end platforms that can run all its online applications. Or at least most of them. Among other things, this so-called "private cloud" relies heavily on Hadoop, the open-source number-crunching platform based on Google's proprietary infrastructure.

According to Shugar, the company's search technologies eat up less than half of the resources served by its cloud, and when search departs for Redmond, other services are sure to fill the hole. "Search is just one of the many services [in use across the platform]. We use it for advertising. We use it for content. We use it for all of our usage logs. It is essentially becoming the data warehouse for all of Yahoo!" he says.

The company now has about 25,000 servers running Hadoop, and Shugar says "search is not the majority user."

Following the July agreement with Microsoft, Hadoop founder Doug Cutting left Yahoo! for the Hadoop-happy startup Cloudera, raising questions about Yahoo!'s future with the open source project. But Shugar says the company may even grow its core Hadoop team. "There will be no backing away from Hadoop. You may even see an acceleration."

Yahoo! is Hadoop's largest contributor, and the company will join the community as it descends on midtown Manhattan later this week for Hadoop World, the platform's first east coast developer conference.

Shugar says Yahoo! is still at work on an SQL-like Hadoop programming language that dovetails with a lower-level language known as Pig. Appropriately enough, the Yahooligans call it Pig SQL.

Yes, there's already an SQL-like Hadoop language: Hive, developed at Facebook. But much to the Facebook's chagrin, Yahoo! feels the need to build a language that better suits its existing Hadoop infrastructure. "We have this Pig stack. We have a lot of things built on this Pig stack, including an optimizer," Shugar says. "We want to keep that stack streamlined."

That said, the new language will be open-sourced.

One thing Yahoo! will not do - at least for the foreseeable future - is open up its infrastructure to external developers, as Google, Microsoft, and Amazon have done. Its private cloud will remain private. But, Shugar points out, Yahoo! is juicing developers through its open source contributions. Despite the media-fueled notion that it's nothing but an online advertising outfit, he insists, Yahoo! is still a technology company. ®

Reducing the cost and complexity of web vulnerability management

More from The Register

next story
Wanna keep your data for 1,000 YEARS? No? Hard luck, HDS wants you to anyway
Combine Blu-ray and M-DISC and you get this monster
US boffins demo 'twisted radio' mux
OAM takes wireless signals to 32 Gbps
Apple flops out 2FA for iCloud in bid to stop future nude selfie leaks
Millions of 4chan users howl with laughter as Cupertino slams stable door
Students playing with impressive racks? Yes, it's cluster comp time
The most comprehensive coverage the world has ever seen. Ever
Run little spreadsheet, run! IBM's Watson is coming to gobble you up
Big Blue's big super's big appetite for big data in big clouds for big analytics
Seagate's triple-headed Cerberus could SAVE the DISK WORLD
... and possibly bring us even more HAMR time. Yay!
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Beginner's guide to SSL certificates
De-mystify the technology involved and give you the information you need to make the best decision when considering your online security options.