Feeds

Facebook's new comms: 'our largest ever engineering project'

15 whole engineers tackle Hbase!

  • alert
  • submit to reddit

High performance access to file storage

The creation of Facebook's new messaging system was the company's largest-ever engineering project, according to director of engineering Andrew Bosworth.

The project spanned more than a year, according to company founder Mark Zuckerberg, and it included the roll-out of a new distributed database platform. The system uses HBase, the open source incarnation of Google's BigTable platform that was originally built by the semantic search outfit Powerset, now owned by Microsoft, as part of the Apache Hadoop project, which mimics multiple pieces of Google's proprietary infrastructure.

It's telling, however, that the largest engineering project in the history of Facebook was handled by a mere 15 engineers.

The system is designed to provide a single interface for handling email, IM, text messages, and on-site Facebook messages, and it includes a single archive for all these various communications. To accomodate this, Bosworth said, the company needed a new database infrastructure. The company was already using MySQL as the primary repository for user data, the open source Cassandra platform for inbox search, the Hadoop-friendly SQL-like language known as Hive for analytics, and the proprietary Haystack for photos. But Hbase provided something different.

"In order to support this really cool and very deep scenario...we needed to rebuild our [messaging] infrastructure." Bosworth said. "Over a year ago, we started looking at storage systems...We weren't sure of the trade-offs [with Cassandra]. We tested MySQL, but we weren't sure it could perform with long-tail data. So we invested in HBase." Hbase is built atop HDFS, the Hadoop File System, an open source incarnation of the old Google File System (GFS).

"Because we want to expose the long-tail of your conversation history really easily, a log-based storage system like HBase makes the most sense," Bosworth told The Reg. "Cassandra — which we love and we built — has some trade-offs around consistency. Because we want this to be real-time — so you always know what messages you're getting — we didn't like those trade-offs."

But the platform also taps Haystack — the existing photo infrastructure — to provide support for email attachments.

Asked if Facebook has any intention of standardizing on a single database platform — if juggling so many platforms would eventually cause unwanted issues — Bosworth told us that for the time being, the company intends to use separate platforms for separate tasks. "With Facebook's technology stack in general, we've really tried to use the right technology for the problem we're solving," he said. "You can get into trouble over-standardizing the technology.

"You build a round hole because you have a round peg. But then you get a square peg and you can't fit it into the hole. We approach things differently"

Also differently from Google.

"We have small, really quick engineering teams. This is the biggest engineering team we've ever built around a new product, and it's still only 15 engineers." ®

High performance access to file storage

More from The Register

next story
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Android engineer: We DIDN'T copy Apple OR follow Samsung's orders
Veep testifies for Samsung during Apple patent trial
OpenSSL Heartbleed: Bloody nose for open-source bleeding hearts
Bloke behind the cockup says not enough people are helping crucial crypto project
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Windows XP still has 27 per cent market share on its deathbed
Windows 7 making some gains on XP Death Day
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
US taxman blows Win XP deadline, must now spend millions on custom support
Gov't IT likened to 'a Model T with a lot of things on top of it'
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.