Bankrupt WorldCom sees major backbone problems
Outage blamed on 'routing anomaly'
WorldCom Inc, which is said to carry as much as 70% of the internet's traffic, suffered an outage followed by a sustained period of severe latency yesterday, apparently after a software issue occurred during routine maintenance,Kevin Murphy writes
The outage, referred to as a "routing anomaly", occurred on the UUNet network on the US east coast at 8am local time and service was mostly restored by 2.30pm, according to reports. It appears that misconfigured border router tables may be to blame.
While only WorldCom customers lost their connections, overall internet performance was affected.
InternetHealthReport.com, a service of Keynote Systems Inc, reported that the mean latency on routes between UUNet and its peers was as high as two seconds yesterday. Keynote classes latency over 180 milliseconds as "critical".
WorldCom spokesperson Jennifer Baker said in a statement: "Today WorldCom experienced an issue on its IP network, affecting approximately 20% of our US IP customer base. Service to our customers has been restored. A preliminary investigation indicates there was a route table issue."
According to UUNet's web site, maintenance was scheduled for POPs in over a dozen US cities between 3am and 6am yesterday, so it is conceivable it was during that time the problem occurred. The company performs maintenance a few times a week. A WorldCom spokesperson did not return a call for comment
Reuters reported that WorldCom memos to customers indicated the problems originated in Los Angeles and Washington DC, although the knock-on effect of rerouting traffic around the black-outs dragged down the whole network. Both LA and Washington were on UUNet's Thursday morning maintenance list.
The problem is an embarrassment for WorldCom at a time when it can do without any more red faces. The company is in Chapter 11 bankruptcy protection while it restructures after revealing the largest accounting scandal in history, and its former executives have become poster children for those who condemn corporate greed.
The company laid off 17,000 people starting in June when the news of the accounting irregularities first broke. While new CEO John Sidgmore assured customers "services are in no way affected" by the layoffs and restructuring, anecdotal evidence suggests many customers, have been buying backup connectivity from other providers.
It's not just enterprise customers that are looking for contingency plans. According to WorldCom's rivals, branded ISPs have been increasingly buying dialup capacity from alternate providers as backup since WorldCom's woes emerged.