Transatlantic link typo by Sweden's Telia broke Cloudflare in the US
Whose fault was it? We have a pretty good Ikea
A fat-fingered human accidentally broke a transatlantic internet backbone that knackered Cloudflare's content delivery network in the US.
Cloudflare – which props up loads of big names online – said that over the course of about fifteen minutes on Tuesday morning, it suffered a slowdown in traffic, meaning connections to websites were hampered. The issue was first reported around 0745 Pacific Time (1445 UTC) with a fix deployed to defuse the situation by 0759, and the problems declared fully resolved by 0853.
"The issue is related to a specific transit provider and we are working on temporarily disabling this provider to route around the issue," Cloudflare said at the time.
In addition to connections in North America, the outage caused problems for Cloudflare services in South America, Europe, Asia, Africa, and the Oceania region.
The bungling transit provider in question was quickly fingered as Telia, a Swedish tech giant that ships internet traffic around the world, from Europe to the US to Asia.
Today's packet loss award goes to Telia. pic.twitter.com/FwFYUS28vz— Hector Martin (@marcan42) May 2, 2017
In a note to customers today, Telia said the knackered performance was the result of a misconfiguration by one of its workers. "The issue was caused by a human mistake while configuring backbone links," Telia told punters.
"As soon as the issue was identified, the configuration was corrected and the transatlantic traffic is now running as expected."
Cloudflare confirmed to The Register that Telia's woes were indeed the cause of its brief downtime – and claimed that a number of Cloudflare's competitors were also affected by the gaffe. We pinged several competing CDN providers for comment on this, but at the time of publication have yet to receive any word of disruptions related to the Telia fumble.
Curiously, the rival CDNs' status pages are quiet on any disruption: Rackspace, Level 3, and Akamai's status pages, for example, show no record of any blips. Incapsula shows some transit issues around 0926 PT, hours after Cloudflare's stumble. ®