Feeds

Level 3's UPS burnout sends websites down in flames

London data centre hit by 3.35am scorcher

Next gen security for virtualised datacentres

UK sysadmins woke up to a headache this morning after a major power cut at Level 3's data centre in Braham Street, East London, lasted for approximately five hours.

The electrical fault developed at 3.35am, according to an error report seen by The Register, and knocked dozens of companies offline. The facility provides space for server hosting firms and connectivity to the internet's traffic backbone.

From the report it looks as though the failure was caused by a busbar blowing in the Uninterruptible Power Supply (UPS) system that is supposed to protect computers and network equipment from unexpected electrical failures.

Busbars are strips of copper or aluminium that distribute electricity through the data centre. The fault cut the AC supply, leaving just a DC current to machines. The paperwork states:

04:14 GMT - Field Services advised that a UPS system failure has occurred in the London (Braham), England site. Specifically, the switchboard on the UPS system is supplying DC power to the equipment at the site using commercial power; however, the AC power supply to the equipment is down. Technicians are working on the UPS system to restore AC power to the affected equipment.

05:19 GMT - Power technicians have discovered 4 bus-bars scorched on the UPS system which caused the AC power failure. Presently, the power technicians are collaborating on a work-around to restore AC power to the site.

Technicians worked to restore AC power to the affected equipment, and started to succeed by 7.30am. The power was restored by 9am.

ISP LCHost, whose internet traffic is routed through the Level 3 data centre, was among businesses hit by the outage. In a Facebook post, the company wrote earlier this morning:

There is an outage for all ADSL customers at present due to the power failure in Level 3 Braham Street where our onward connection to BT's network passes through. As a result, ADSL customers will be unable to connect at this time.

Level 3 engineers are working to restore power and in the meantime we are examining options for rerouting the connection in the event that the power restoration cannot be achieved in a reasonable time frame.

The ISP added that its connectivity was back to normal by 8.50am.

We've left messages with Level 3, one of the world's top-tier providers of global internet connectivity, to ask for more details. ®

Updated to add

A Level 3 spokeswoman confirmed the data centre suffered a power outage that was "resolved by the team of on-site power technicians".

The essential guide to IT transformation

More from The Register

next story
6 Obvious Reasons Why Facebook Will Ban This Article (Thank God)
Clampdown on clickbait ... and El Reg is OK with this
So, Apple won't sell cheap kit? Prepare the iOS garden wall WRECKING BALL
It can throw the low cost race if it looks to the cloud
EE fails to apologise for HUGE T-Mobile outage that hit Brits on Friday
Customer: 'Please change your name to occasionally somewhere'
Time Warner Cable customers SQUEAL as US network goes offline
A rude awakening: North Americans greeted with outage drama
We need less U.S. in our WWW – Euro digital chief Steelie Neelie
EC moves to shift status quo at Internet Governance Forum
BT customers face broadband and landline price hikes
Poor punters won't be affected, telecoms giant claims
prev story

Whitepapers

Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Advanced data protection for your virtualized environments
Find a natural fit for optimizing protection for the often resource-constrained data protection process found in virtual environments.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.