Feeds

Twitter jilts Ruby for Scala

A new love on the back-end

Boost IT visibility and business value

Web 2.0 Expo Famously, when Twitter's Web2.0rhea app was suddenly embraced by digerati+dog in late 2007, its original Ruby on Rails architecture had more than a little trouble keeping up the endless stream of digital solipsism.

But in mid-2008, the web's startup-of-the-moment quietly ported some of its core code to Scala, and according to Twitter developer Alex Payne, the switch should stand as a lesson to cutting-edge coders everywhere.

Addressing developers at the annual Web 2.0 Expo in San Francisco, California, Payne argued that Scala - not Ruby on Rails or PHP or Python - should be the language of choice for the modern web startup.

Today, Payne said, most of the hip dev set codes in Ruby or PHP or Python because they're perceived as "agile" languages suited to quick site changes - but also because the deverati grew bored with languages like Java and C++. The trouble, he insisted, is that the so-called Web 2.0 languages aren't always as efficient as they need to be, especially in an economic environment threatening to bring Web 2.0 to its knees.

"Investors now want to know that you're not going to be paying tons and tons of money for servers because you decided to build in a stack that isn't as efficient as possible," he said.

"In a down market, you may have to make choices between technologies you love to work with and technologies that will keep the lights on. The question is... whether we can afford to build things that we love with tools that we love. The answer is 'Yes'. But it means adopting a different set of tools."

For Payne, who oversees Twitter's API development, your ticket to the web promised land is Scala, the next-generation language developed by Martin Odersky at Switzerland's École Polytechnique Fédérale de Lausanne.

Sometime in 2007, Twitter came to the realization that its Ruby-based message queuing system couldn't keep up with all that Web2.0rhea. "It grew with us for a number of months, and then, at a certain point, it just hit a wall," Payne said. "And that wall was not so much it code but in limitations in Ruby virtual machine. There's a lot of things that Ruby is great at, but long running processes? Particularly memory intensive ones? Not so much."

By mid-2008, one of these Ruby message queues completely crashed and developers needed two and a half hours to shove the dropped Tweets back through the system. When your game is micro-blogging, that's a lifetime. Then, in his spare time, one developer ported the code to Scala. According to Payne, the Scala queue could process the same message backlog in 20 seconds.

The new system was in place before the end of last year, Payne told The Reg, just in time for the Twittering chaos of the Barack Obama inauguration.

By the end of this year, Payne said, Twitter hopes to have its entire middleware infrastructure and its APIs ported to the new language. Ruby will remain, but only on the front end. "We're still happy with Rails for building user facing features... performance-wise, it's fine for people clicking around web pages. It's the heavy lifting, asynchronous processing type of stuff that we've moved away from."

For this sort of heavy lifting, languages like Java and C++ may be as fast as Scala. But Payne wants more than speed. He prefers Scala because it combines functional and object-oriented techniques. "Functions are objects, and objects are functions," he said. "Scala asks can you have the best of these two worlds together? And you can."

But he also prefers Scala because it's, well, beautiful. "It's a fuzzy thing. But we like writing beautiful code, code that you're proud of, code that you can show to non-programmers and they get it."

When Twitter made the Scala switch, it wasn't using Java Virtual Machines (JVMs). But, Payne argues, the language is that much more attractive if you're already running Java.

"The biggest selling point for certain organizations is that Scala is 100 per cent compatible with your existing JVM code. There's no wrappers. There's no bullshit. You just load up your JVM code and call in to it, and there's no performance penalty. If you have a big investment in Java infrastructure, Scala is the ideal language."

But if you don't, Payne will tell you much the same thing. Clearly, he's in love. And he's pitching his new Scala book. And his new Scala book is published by O'Reilly. And O'Reilly runs the Web 2.0 Expo. ®

The Essential Guide to IT Transformation

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
KDE releases ice-cream coloured Plasma 5 just in time for summer
Melty but refreshing - popular rival to Mint's Cinnamon's still a work in progress
Leaked Windows Phone 8.1 Update specs tease details of Nokia's next mobes
New screen sizes, dual SIMs, voice over LTE, and more
Another day, another Firefox: Version 31 is upon us ALREADY
Web devs, Mozilla really wants you to like this one
Put down that Oracle database patch: It could cost $23,000 per CPU
On-by-default INMEMORY tech a boon for developers ... as long as they can afford it
Secure microkernel that uses maths to be 'bug free' goes open source
Hacker-repelling, drone-protecting code will soon be yours to tweak as you see fit
Mozilla keeps its Beard, hopes anti-gay marriage troubles are now over
Plenty on new CEO's todo list – starting with Firefox's slipping grasp
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Build a business case: developing custom apps
Learn how to maximize the value of custom applications by accelerating and simplifying their development.