Feeds

Level 3's UPS burnout sends websites down in flames

London data centre hit by 3.35am scorcher

Internet Security Threat Report 2014

UK sysadmins woke up to a headache this morning after a major power cut at Level 3's data centre in Braham Street, East London, lasted for approximately five hours.

The electrical fault developed at 3.35am, according to an error report seen by The Register, and knocked dozens of companies offline. The facility provides space for server hosting firms and connectivity to the internet's traffic backbone.

From the report it looks as though the failure was caused by a busbar blowing in the Uninterruptible Power Supply (UPS) system that is supposed to protect computers and network equipment from unexpected electrical failures.

Busbars are strips of copper or aluminium that distribute electricity through the data centre. The fault cut the AC supply, leaving just a DC current to machines. The paperwork states:

04:14 GMT - Field Services advised that a UPS system failure has occurred in the London (Braham), England site. Specifically, the switchboard on the UPS system is supplying DC power to the equipment at the site using commercial power; however, the AC power supply to the equipment is down. Technicians are working on the UPS system to restore AC power to the affected equipment.

05:19 GMT - Power technicians have discovered 4 bus-bars scorched on the UPS system which caused the AC power failure. Presently, the power technicians are collaborating on a work-around to restore AC power to the site.

Technicians worked to restore AC power to the affected equipment, and started to succeed by 7.30am. The power was restored by 9am.

ISP LCHost, whose internet traffic is routed through the Level 3 data centre, was among businesses hit by the outage. In a Facebook post, the company wrote earlier this morning:

There is an outage for all ADSL customers at present due to the power failure in Level 3 Braham Street where our onward connection to BT's network passes through. As a result, ADSL customers will be unable to connect at this time.

Level 3 engineers are working to restore power and in the meantime we are examining options for rerouting the connection in the event that the power restoration cannot be achieved in a reasonable time frame.

The ISP added that its connectivity was back to normal by 8.50am.

We've left messages with Level 3, one of the world's top-tier providers of global internet connectivity, to ask for more details. ®

Updated to add

A Level 3 spokeswoman confirmed the data centre suffered a power outage that was "resolved by the team of on-site power technicians".

Intelligent flash storage arrays

More from The Register

next story
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
Broadband sellers in the UK are UP TO no good, says Which?
Speedy network claims only apply to 10% of customers
Virgin Media struck dumb by NATIONWIDE packet loss balls-up
Turning it off and on again fixes glitch 12 HOURS LATER
Fujitsu CTO: We'll be 3D-printing tech execs in 15 years
Fleshy techie disses network neutrality, helmet-less motorcyclists
Facebook, working on Facebook at Work, works on Facebook. At Work
You don't want your cat or drunk pics at the office
Soz, web devs: Google snatches its Wallet off the table
Killing off web service in 3 months... but app-happy bonkers are fine
Ofcom tackles complaint over Premier League footie TV rights
Virgin Media: UK fans pay the most for the fewest matches
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
prev story

Whitepapers

Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Protecting users from Firesheep and other Sidejacking attacks with SSL
Discussing the vulnerabilities inherent in Wi-Fi networks, and how using TLS/SSL for your entire site will assure security.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.