Facebook shares own tools to trap bugs before they break code

Test management and debugging at scale become a bit less daunting

Facebook device testing

Facebook on Wednesday plans to introduce a set of open source developer tools to streamline app development testing and bug hunting.

At the Dev Tools @Scale conference in London, Facebook intends to show off: OneWorld, a system for testing software across multiple devices and environments; Jupiter, a tool for matching backend jobs with computing resources; and AL, a declarative coding language for writing custom bug checkers that work with Facebook's open source static analyzer, Infer.

In a phone interview with The Register, engineering director Bryan O'Sullivan said that while Facebook may be known for high-profile projects like Oculus and Aquila, the company has dozens of engineers working on ways to make software development more efficient.

OneWorld embodies that goal. Designed to complement Facebook's mobile device lab, which allows developers to run tests on thousands of mobile devices at once, OneWorld provides an API to manage remote testing resources, such as mobile phones, browsers, and emulators.

At Facebook, OneWorld coordinates something like 20,000 endpoint resources and processes over a million jobs per day.

"This is a system that aims at helping engineers work with Android devices and iPhones and the emulators that simulate these things, and browser well," said O'Sullivan. "Our aim is making it as straightforward to deal with these resources as possible."

OneWorld consists of a runtime worker service (a daemon that runs on machines connecting to remote resources), a scheduler (Jupiter, in this case), and a satellite service that allows engineers to connect a local resource to the One World cloud.

"These different environments, they tend to a bit quirky and flaky," said O'Sullivan. "Prior to OneWorld, engineers would often get or be given hardware and it didn't work, or they couldn't get what they needed. OneWorld hides all these details."

Jupiter works in conjunction with OneWorld and also supports other jobs submitted by software engineers, such as building new packages, installing app dependencies, and running tests.

The service attempts to match requests for work with the computing resources available to handle that work and keeps track of job processes to ensure efficient resource allocation.

Facebook developers write a lot of tests to make certain their code works properly. Jupiter helps those tests get submitted and processed as quickly as possible. "With Jupiter, we went from taking hundreds of seconds to milliseconds in scheduling a job," explained Adriana Liborio, software engineer at Facebook.

AL, meanwhile, is a new declarative language for writing custom bug checkers for Infer, Facebook's static code analyzer, a tool for finding bugs in source code.

"Infer can understand what source code does without running it and tries to find inconsistencies," said Jules Villard, a software engineer at Facebook. He said that static analysis is complementary to testing and typically catches thousands of issues every month before they make into the apps running on the phones of Facebook users.

Infer attempts to prove the code is bug-free, though doesn't produce proofs, Villard said.

"It can find bugs in parts of the code that an engineer won't necessarily be able to think of," said O'Sullivan. "That's incredibly valuable because it's exhaustive."

AL, which stands for AST (abstract syntax tree) Language, provides a means to define a bug checker without tweaking Infer's source code. It makes extending Infer less complicated.

Using AL, a developer could, for example, declare the following rule and then check code for compliance: "A property containing the word 'delegate' but not containing the word 'queue' should not be declared strong."

These tools, said O'Sullivan, "save very large amounts of time and they directly contribute to the quality of the software Facebook engineers are shipping to two billion people a month."

They may yet prove useful to other organizations. ®

Sponsored: Minds Mastering Machines - Call for papers now open




Biting the hand that feeds IT © 1998–2018