Data Centre


The Social Network™ releases its data networking code

Open/R distributed router software goes open source, with Juniper, Arista aboard

By Richard Chirgwin


Facebook has sent another shiver running up Cisco's spine, by releasing the code it uses for packet routing.

Open/R, its now-open source routing platform, runs Facebook's backbone and data centre networks. The Social Network™ first promised to release the platform in May 2017.

In the post that announced the release, Facebook said it began developing Open/R for its Terragraph wireless system, but since applied it to its global fibre network, adding: “we are even starting to roll it out into our data center fabrics, running inside FBOSS and on our Open Compute Project networking hardware like Wedge 100.”

Apart from Facebook's internal platforms, the company launched Open/R with support from third-party vendors including:

Facebook said in the post that while working on Open/R with Terragraph, it decided it needed a distributed networking platform to get over the shortcomings of both Layer 2 protocols for large networks (lacking resiliency) and Layer 3 protocols (“poorly suited to highly variable and dynamic outdoor conditions”).

“Thus, we designed Open/R to be the distributed networking platform on top of the Terragraph network that could cope with its large scale and rapid changes, and allow for fast extensibility”, the post said.

Those features also made Open/R suitable to control traffic and routing across the Express Backbone it built in May.

“Instead of using a traditional routing protocol such as IS-IS, we decided to introduce Open/R into the Express Backbone as the distributed control plane for the network, where it would work together with a centralised controller. Here, Open/R provides basic connectivity in a network, fast reactions to network events, and a "distributed information bus" to export new information to the controller and agents on the network nodes.”

Open/R uses a database as the control plane rather than a routing protocol (as noted on slide 10 of this IETF presentation by Tom Herbert of Quantonium and Petr Lapukhov – a co-author of the announcement – describing a proposed IPv6 location identifier standard).

Open/R architecture

The image below shows the key architectural components.

Open/R architecture. Click to embiggen.

The KV-STORE – key-value store – is replicated around the network for distributed communications and state replication; Spark handles neighbour discovery and reporting.

LinkMonitor's role is probably self-explanatory, PrefixManager handles “automatic prefix suballocation”, Decision is the route information computer, FIB is the route programming proxy, and Platform “implements route programming and interface discovery logic for the target hardware platform.”

So engineers don't need to put live networks at risk, Open/R offers an emulation environment: a systemd-nspawn wrapper. This, Facebook wrote, “starts thousands of Linux containers and forms a mesh of tunnels among them with a single command”, so engineers can test the impact of latency, jitter, and packet loss.

The distributed state database lets an administrator view current network state from any node on the network, watching over activity like nodes connecting, adjacencies tearing down, or metric values changing.

For functions that need a centralised controller, the post said, “Open/R provides APIs allowing remote agents to learn the link state or subscribe to database updates, such as notifications of a link capacity change”.

Other features Facebook listed include:

The post also said Open/R includes “many features similar to those found in IS-IS (intermediate system to intermediate system) and OSPF (open shortest path first).

Open/R's GitHub repository is here. ®

Sign up to our NewsletterGet IT in your inbox daily


More from The Register

Facebook suggests mm-wave spectrum should be free in Australia

Hints at multi-gig wireless, which should get nbn™ - and government - feeling nervous

Billionaire Buffett's Berkshire liquidates $2.1bn stake in Oracle – months after buying the shares

Bill Gates' bridge chum's firm buys Red Hat stock

ZX Spectrum reboot scandal man sits on Steve Bannon design tech shindig committee

RCL chief David Levy is member of ACE2018 committee

Ofcom: More spectrum for all the good boys and girls. Except you, EE. You've had your fill

UK mobile networks had better open their wallets

Vodafone sues Ofcom to reclaim 'overpaid' mobe spectrum fees

EE set it up, now Voda's shooting for goal

Is this a wind-up? Planet Computers boss calls time on ZX Spectrum reboot firm

Lots of unanswered questions remain – and nobody's talking

Facebook's new always-listening home appliance kit Portal doesn't do Facebook

Trust us, pleads the Zuck

Golly 4G whizz: Vodafone caught using a cheeky bit of Three UK's spectrum

Three: 'A number of customers' would've seen performance choke due to 'error'

Oracle gets busy with Lazy FPU fix, adds more CPU Spectre-protectors

Oracle Linux and VM get their innoculations

Old bugs, new bugs, red bugs … yes, it's Oracle mega-update day again

Out of 284 flaws, 33 are rated critical. Big Red admins have big patches ahead