Feeds

Twitter survives election after Ruby-to-Java move

Peak traffic of 874,560 Tweets per minute without Fail Whale coming up for air

5 things you didn’t know about cloud backup

Micro-blogging site Twitter experienced record traffic as the results of the 2012 US Presidential election were announced on Tuesday night, but the service never faltered despite the increased load – something Twitter engineers credit to the company's move from Ruby to Java for its backend software.

According to a blog post by Mazen Rawashdeh, Twitter's VP of infrastructure operations engineering, Twitter users posted an average of 9,965 messages per second between the hours of 8:11pm and 9:11pm Pacific Time.

During a single second at 8:20pm, Twitter users produced 15,107 new posts, Rawashdeh writes, and during the peak traffic period of the evening they generated 874,560 posts in a single minute.

Such numbers are unusual for Twitter, Rawashdeh says, and they represent an evolution in how customers use the service. While in the past Twitter has experienced brief traffic spikes to mark particular events, such as New Year's Eve or the close of a sporting event, this was the first time such heavy traffic lasted for such a sustained period. It was also the highest volume of traffic during an election since Twitter launched.

In the past, Twitter users might have expected to experience problems when the service is under such heavy load. Early in Twitter's existence, service outages were so common that the company's whimsical "fail whale" error screen became a cultural icon for the Web 2.0 crowd.

But Tuesday night's election-driven traffic went off with nary a hitch, which Rawashdeh attributes to Twitter's ongoing migration from backend software written in Ruby and the Ruby on Rails framework to a new software stack built on the Java Virtual Machine (JVM).

Fail whale

Fail Whale sightings were a lot more common when Twitter ran on Ruby. With Java, not so much

Twitter first began stepping away from Ruby in 2008, when the company's Ruby-based message queuing system "hit a wall," in the words of former Twitter developer Alex Payne.

"There's a lot of things that Ruby is great at," Payne said at the time, "but long running processes? Particularly memory intensive ones? Not so much."

Twitter's solution was to migrate some of its Ruby code to a new server stack running on the JVM. Initially, the company's development team avoided stock Java in favor of Scala, an alternative JVM language that combines aspects of object-oriented and functional programming. Today, Twitter's software is built from a mix of Scala and ordinary Java code.

A few Twitter services still run on Ruby, too, but according to Rawashdeh these are used increasingly less often. Notably, he says Twitter has now reconfigured its system so that traffic from mobile devices never touches any Ruby-based software at all.

Where Twitter does use Ruby, it deploys the code on a custom, highly optimized version of the Ruby runtime designed to manage memory more efficiently when executing long-running processes.

None of this will be welcome news to the army of fanatical Ruby developers who believe the language's syntax, its high developer productivity, and its overall philosophy far outweigh any performance disadvantage it might have compared to other languages.

But for Twitter, results are what matter. "The bottom line: No matter when, where or how people use Twitter, we need to remain accessible 24/7, around the world," Rawashdeh writes. "We're hard at work delivering on that vision."

They had better be. Judging by the torrential outpouring of comment on all things related to the recent election, Twitter can only expect its average traffic load to keep climbing as Barack Obama enters his second term as US President. ®

5 things you didn’t know about cloud backup

More from The Register

next story
6 Obvious Reasons Why Facebook Will Ban This Article (Thank God)
Clampdown on clickbait ... and El Reg is OK with this
So, Apple won't sell cheap kit? Prepare the iOS garden wall WRECKING BALL
It can throw the low cost race if it looks to the cloud
EE fails to apologise for HUGE T-Mobile outage that hit Brits on Friday
Customer: 'Please change your name to occasionally somewhere'
Time Warner Cable customers SQUEAL as US network goes offline
A rude awakening: North Americans greeted with outage drama
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
We need less U.S. in our WWW – Euro digital chief Steelie Neelie
EC moves to shift status quo at Internet Governance Forum
BT customers face broadband and landline price hikes
Poor punters won't be affected, telecoms giant claims
prev story

Whitepapers

5 things you didn’t know about cloud backup
IT departments are embracing cloud backup, but there’s a lot you need to know before choosing a service provider. Learn all the critical things you need to know.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Backing up Big Data
Solving backup challenges and “protect everything from everywhere,” as we move into the era of big data management and the adoption of BYOD.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?