Feeds

Yahoo! open sources uber web server

'400-terabytes-a-day' Traffic Server lands at Apache

Build a business case: developing custom apps

Yahoo! has open sourced the back-end software platform that underpins the company's webmail client and countless other applications offered up across its sweeping web portal.

Known as Traffic Server, the platform handles general edge caching, edge processing, and load balancing at Yahoo!, but it's also used to manage traffic on the company's internal storage and server-virtualization services.

"It's used at the edge, but it also gets used almost like an application server," Chuck Neerdaels, Yahoo! vp of content storage, delivery, and edge, tells The Reg. It serves up the latest version of Yahoo! Mail, for instance, which the company calls "Candy Gram."

Acquired with Yahoo!'s purchase of Inktomi, Traffic Server has been in active use at the two companies for the past eight years. According to Yahoo!, it now handles 30,000 requests per second, serving 30 billion Web objects and 400 terabytes of data a day.

"It is a very mature, very reliable piece of technology," says Shelton Shugar, Yahoo!'s senior vp of cloud computing. "In some form, it supports more than half of Yahoo!'s traffic."

The company donated a version of the platform to The Apache Software Foundation last week through the Apache Incubator program. "This is part of our overhaul strategy to open source cloud services that are mature and not laden with Yahoo!-specific stuff that wouldn't make sense for open source," says Shugar.

In June, the company open sourced its internal Hadoop distro, an internet-scale distributed data-crunching platform based the Apache project of the same name.

Neerdaels says Traffic Server was originally designed as a proxy cache. But it's Yahoo!'s go-to tool for http session management, and it includes an API for tweaking content all the way down at the protocol level. "You can poke around with various headers and inject content and direct client requests to different backends, all through a relatively clean API," he says.

Neerdaels and Shuger also describe Traffic Server as an "extensible framework" that lets you tweak the architecture according to the task at hand. And, according to the company, it suits web operations both large and small. "It's simple enough for a small operation to pick it up quickly," says Neerdaels. "But the big players can pick it up and it will scale to meet their needs pretty impressively."

Those 400 terabytes are served up from between 100 and 150 "commodity" machines. ®

Build a business case: developing custom apps

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
KDE releases ice-cream coloured Plasma 5 just in time for summer
Melty but refreshing - popular rival to Mint's Cinnamon's still a work in progress
Leaked Windows Phone 8.1 Update specs tease details of Nokia's next mobes
New screen sizes, dual SIMs, voice over LTE, and more
Another day, another Firefox: Version 31 is upon us ALREADY
Web devs, Mozilla really wants you to like this one
Put down that Oracle database patch: It could cost $23,000 per CPU
On-by-default INMEMORY tech a boon for developers ... as long as they can afford it
Secure microkernel that uses maths to be 'bug free' goes open source
Hacker-repelling, drone-protecting code will soon be yours to tweak as you see fit
Mozilla keeps its Beard, hopes anti-gay marriage troubles are now over
Plenty on new CEO's todo list – starting with Firefox's slipping grasp
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Backing up Big Data
Solving backup challenges and “protect everything from everywhere,” as we move into the era of big data management and the adoption of BYOD.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.