Telecity London data centre outage borks VoIP, websites, AWS...
LINX reports sudden sharp traffic drop, Amazon Direct Connect goes TITSUP
Update 2 Telecity has suffered a major outage at one of its London data centres this afternoon, which knocked out a whole host of VoIP firms' services, made Amazon wobble and borked its Direct Connect service.
A source told The Register that the outage, which happened at around 2pm, knocked out four floors at Telecity's Sovereign House data centre, which is located around the corner from London's Canary Wharf financial district.
The Register understands both primary and backup power supplies went down, potentially affecting thousands of customers.
The firm, which was bought out by Equinix, secured regulatory approval for the buyout earlier this month - but was ordered to divest five of its eight London data centres.
Tim Stephenson, head of devops at Allies Computing, told El Reg: "We use AWS EU-WEST-1 to host a number of our web services, and since around 2pm have been seeing intermittent connectivity fails to servers and ELB instances running in the region. Traffic routing in from transatlantic USA connections seems to be affected more than European or UK connections, but there’s not that much consistency in terms of impact."
He added: "It would indeed appear that Telecity dropping offline has more impact on EU-WEST-1 than Amazon appear to be prepared to admit, as our services don’t use AWS Direct Connect and yet are still being affected."
Another source told us their firm's Office 365 and Lync/Skype connections were affected, along with their AWS Direct Connect dropping all their VPC tunnels.
The outage coincided with a noticeable dip in internet traffic across the London Internet Exchange (LINX).
Did the Telecity outage affect the UK's total internet traffic? Screencap taken at 1520GMT today
Many small firms took to Twitter to reassure irate customers.
Our VOIP phone provider is experiencing technical difficulties. Our phones are offline all other channels are working as normal.— Nublue (@Nublue) November 17, 2015
@Will_Ogden Correct - The issue is due to a major power outage in the Telecity data centre. Engineers are on-site and working on a solution— dotmailer (@dotmailer) November 17, 2015
We are aware of a partial power outage in Telecity Sovereign House. Our router is still up, but this is by no means universal!— Safehosts Cheltenham (@SafehostsDC) November 17, 2015
The outage even made Amazon wobble, as the Jeff Bezos-owned company's status page very quietly admitted:
Amazon's AWS Direct Connect service was hit, causing AWS' EU-WEST-1 server to suffer
Telecity refused to comment when The Register phoned them to ask what had happened. ®
Are you or your firm affected by the Telecity outage? Contact us: email@example.com or phone the London newsdesk on +44 (0)20 3770 3147.
Sponsored: Becoming a Pragmatic Security Leader