MessageLabs in partial email outage
Email slowed to snail mail for eight hours
Posted in Music and Media, 24th May 2002 11:57 GMT
Free whitepaper – Dell/EMC CX4 and Dell PowerEdge blades
A substantial number of virus filtering firm MessageLabs' customers experienced severe delays in sending and receiving email yesterday because of network routing problems.
Customers, mainly in the London area, connected to two of the managed service providers 20 server racks would have noticed slow email receipt and difficult sending messages from around midday until 7.45pm yesterday, when it was able to resolve the problem.
Alex Shipp, chief antivirus technologist at MessageLabs, explained: "The problem was caused by network misconfiguration at one of our telecommunication providers. This caused SMTP sessions to certain IP ranges to initiate, but to fail after a while, which meant that mail to and from those IP ranges was very slow."
"Lower level routing protocols could not automatically discover the problem (the route worked *most* of the time) and so did not automatically switch routes," he added.
None of the messages were lost but the delays to important messages caused consternation at a number of MessageLabs' customers, who contacted us about the problem.
"This is supposed to be a high-availability, business critical service. There must be a lot of pissed off people about who were expecting important emails to arrive yesterday afternoon," said one network admin who contacted us about the issue.
MessageLabs apologised for any inconvenience caused and said it would introduce diagnostic software that would identify the caused of any future partial failure and re-route traffic. Ironically if a particular connection with one of MessageLabs' telco partners had failed entirely traffic would be re-routed automatically and customers would not have noticed any problem. ®

Enabling the Agile Data Center

Dirty, dirty PCs: The X-rated picture guide
Top 500 supers - rise of the Linux quad-cores
Early adopters bloodied by Ubuntu's Karmic Koala
Sign up, sign up for The Register IT security newsletter