Feeds

Yahoo! open sources uber web server

'400-terabytes-a-day' Traffic Server lands at Apache

Internet Security Threat Report 2014

Yahoo! has open sourced the back-end software platform that underpins the company's webmail client and countless other applications offered up across its sweeping web portal.

Known as Traffic Server, the platform handles general edge caching, edge processing, and load balancing at Yahoo!, but it's also used to manage traffic on the company's internal storage and server-virtualization services.

"It's used at the edge, but it also gets used almost like an application server," Chuck Neerdaels, Yahoo! vp of content storage, delivery, and edge, tells The Reg. It serves up the latest version of Yahoo! Mail, for instance, which the company calls "Candy Gram."

Acquired with Yahoo!'s purchase of Inktomi, Traffic Server has been in active use at the two companies for the past eight years. According to Yahoo!, it now handles 30,000 requests per second, serving 30 billion Web objects and 400 terabytes of data a day.

"It is a very mature, very reliable piece of technology," says Shelton Shugar, Yahoo!'s senior vp of cloud computing. "In some form, it supports more than half of Yahoo!'s traffic."

The company donated a version of the platform to The Apache Software Foundation last week through the Apache Incubator program. "This is part of our overhaul strategy to open source cloud services that are mature and not laden with Yahoo!-specific stuff that wouldn't make sense for open source," says Shugar.

In June, the company open sourced its internal Hadoop distro, an internet-scale distributed data-crunching platform based the Apache project of the same name.

Neerdaels says Traffic Server was originally designed as a proxy cache. But it's Yahoo!'s go-to tool for http session management, and it includes an API for tweaking content all the way down at the protocol level. "You can poke around with various headers and inject content and direct client requests to different backends, all through a relatively clean API," he says.

Neerdaels and Shuger also describe Traffic Server as an "extensible framework" that lets you tweak the architecture according to the task at hand. And, according to the company, it suits web operations both large and small. "It's simple enough for a small operation to pick it up quickly," says Neerdaels. "But the big players can pick it up and it will scale to meet their needs pretty impressively."

Those 400 terabytes are served up from between 100 and 150 "commodity" machines. ®

Secure remote control for conventional and virtual desktops

More from The Register

next story
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Sway: Microsoft's new Office app doesn't have an Undo function
Content aggregation, meet the workplace ... oh
Sign off my IT project or I’ll PHONE your MUM
Honestly, it’s a piece of piss
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
NetWare sales revive in China thanks to that man Snowden
If it ain't Microsoft, it's in fashion behind the Great Firewall
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
Admins! Never mind POODLE, there're NEW OpenSSL bugs to splat
Four new patches for open-source crypto libraries
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.