Original URL: https://www.theregister.com/2008/01/23/domain_name_snafu/
Domain name gaffe launches Clearswift clients into e-mail panic
Mission critical app in critical condition
Posted in Channel, 23rd January 2008 20:09 GMT
A domain name snafu at Clearswift, a company that filters email and web pages for objectionable content, wreaked havoc on some of its business customers when admins awoke to find their organizations were unable to send or receive email.
The outage was caused when mimesweeper.biz, the domain where customers' email is routed before being filtered, went off line. It wasn't immediately clear what caused the domain name to crash, but this whois record shows it expired on Dec. 13, suggesting someone forgot to renew its registration. Naughty, naughty.
Alyn Hockey, Clearswift's director of product management, said he was unsure what caused the domain name to fail, but acknowledged the theory about the lapsed registration may be correct.
"Obviously, there will be some work after the fact to make sure it doesn't happen again," he said.
Reports of the problem began trickling in early Wednesday morning in the UK. By evening, he said, support staff had devised a work-around that involved customers putting additional addresses into their email servers' configuration files. If for some reason the first server fails, Hockey said, the change would cause email to be routed to a second or third address rather than bouncing.
Not all customers seem to be aware of solution, and that, they said, is creating big problems. Mike, an admin for a local governmental agency in the UK, is one of the plagued.
"No email has been received since last night, and most of the outbound email has stopped or been bounced under 550," he told El Reg, referring to the numeric error code that frequently indicates a faulty DNS setting. "Basically we will be pursuing serious damages for breach of contract and direct business disruption. At the current time all we have is phone calls as there is no working email."
Clearswift has about 17,000 business customers, and filters content for about 25 million end users. Hockey said only a small proportion of them - perhaps 5 per cent - were affected by the outage.
Out of all the blunders an IT organization can make, forgetting to renew a domain-name registration seems to be among the more common. Last week, Perl.com was caught redirecting users to a porn link farm after ownership of a domain name that was embedded into the site changed hands. And in 2003 someone at Microsoft failed to renew the registration for hotmail.co.uk, an oversight that allowed a Linux hacker to scoop it up.
Here's hoping the cock-ups serve as cautionary tales for all admins that in addition to firewalls, content filtering systems and intrusion detection systems, security also includes more mundane details like renewing domain names. Lose control of them, and you can kiss your security good-bye. ®