Facebook security chief: We're not encrypting everything between our data centers just yet
Sullivan on HTTPS, NSA and paranoia
A couple of weeks ago Facebook scheduled a press powwow with its chief security officer Joe Sullivan to discuss defenses for the social network and its users. Then, a week later, Sullivan's boss made an angry call to the White House to complain about intelligence agents using Facebook as a conduit for spying on people.
"I don’t think anyone who focusses on security has been surprised by the specific things that we've seen," Sullivan told us today about reports stemming from document leaked by NSA whistleblower Ed Snowden. Those documents suggested US intelligence systems were impersonating the Facebook website so as to silently infect victims' PCs with snooping malware.
"As security people, we're paranoid, so we assume all of these things are happening, but when you actually see concrete evidence of an implementation, that moves it from paranoia to professional security advice."
In a way it was better for Facebook that news of the NSA's man-in-the-middle attacks had come out now rather than when the company was much smaller, he said. The social network had been able to hire enough security talent to allow it to work on protecting itself against government-grade operations while maintaining a focus on guarding its users against more run-of-the-mill criminal hackers.
Facebook doesn’t have one security team per se, Sullivan said, but had different units spread around the company watching for attacks on the network's servers as well as on its visitors. Three years ago the company also started challenging its security team to hack its own staff on the tenth month of the year, dubbed Hacktober.
Hacktober was started because the usual training videos and classroom sessions weren't effective, Sullivan said. Instead the security team originally set up a wall of shame, similar to that used at the Defcon security conference, which listed employees who had let their defenses slip.
But that wasn't very effective either, the team found. Staff resented the wall listing, and so the approach was changed from stick to carrot. Now, if staff spot a hacking attempt and reported it, they get a Hacktober t-shirt. Sullivan reported this proved a strong incentive and fostered an inter-company competition to beat the security testers.
Facebook also hires in outside firms for penetration testing, Sullivan explained. In some case the attackers were given access to s small part of Facebook's internal network and asked to escalate their privileges. The internal security team would then pick up on a series of clues until the intrusion was detected and dealt with.
'If data is going through a building or a cable that someone else controls we need to assume the worst'
Recruiting the wider security community, via Facebook's big bounty program, was also a highly effective technique. In the last three years Facebook has paid out more than two million dollars for reports of vulnerabilities in its software, and has hired three researchers who proved particularly adept at finding flaws in its defenses.
The storm caused by the Snowden leaks has had a silver lining for the industry, Sullivan said, in that it had brought erstwhile competing companies together to work on common security issues.
He detailed one case where another company warned Facebook of a dodgy server that was attempting to install malware on PCs used by the social network's employees. Facebook checked to see none of its own staff had visited the site, and found they had not, but saw that the server had tried to infect workers at 50 other tech firms, too. All were warned as well.
In the meantime Facebook is ramping up its security efforts after it turned out US and UK intelligence agencies were tapping into the connections between web giants' data centers to snoop on netizens.
Yet, Facebook is still not encrypting all internal traffic between its off-site data centers: Sullivan blamed weaknesses in encrypting chunks of data flowing through his company's interconnects. Instead, his team had identified key data streams that needed protecting from eavesdroppers, and is locking them off one by one with their own encrypted channels.
All company staff have to turn on two-factor authentication before logging on to Facebook, and Sullivan said he was heartened by how many Facebook users also wanted to use the extra security. About a third of the user base activated two-factor authentication shortly after the security team added it in 2011.
In 2013 the company doubled its encryption key strength to 2,048-bit and has augmented HTTPS with perfect forward secrecy. “With SSL, there’s going to be a single key that opens every car on the highway, and with perfect forward there’s now a different key for each car,” Sullivan said.
For mobile users the company has developed Conceal for Android, a set of Java APIs that encodes large files using cryptographic algorithms from OpenSSL. The company is also investigating third party apps and checking the security of companies that provide it with leased lines to check there are no data leaks – at least, as far as possible.
"We're looking at literally every point in the movement of data and analyzing the risks. If data is going through a building or a cable that someone else controls we need to assume the worst, in the same way that we assume the worst about every one of our employees," Sullivan said.
"I trust everyone I work with, but also assume that they can get malware on their laptop or they might have their spouse held hostage. Everything can go wrong and it's not about trusting people, it's about removing the risk." ®
Sponsored: Optimizing the hybrid cloud