Original URL: https://www.theregister.com/2007/05/01/hotmail_friendly_fire/
Hotmail's antispam measures snuff out legit emails, too
No warning, little recourse
Posted in Security, 1st May 2007 01:50 GMT
Hotmail users and email server admins, beware: you may be unknowingly caught in the crossfire of Microsoft's war on spam. Unintended casualties include legitimate emails from domains with well-established reputations, which are systematically blocked with absolutely no notice and little recourse.
The chief culprit is the inaptly named SmartScreen, a proprietary spam control technology the software Goliath rolled out to great fanfare several years ago. While the filtering mechanism appears to be making some headway in eradicating Viagra come-ons and nasty phishing attempts, the victory comes at a price: an untold number of legitimate emails are blocked with no warning to either sender or intended recipient.
Compounding the indignity in this well-intentioned campaign, these aggrieved admins say, are Microsoft support people who offer canned responses that acknowledge a domain's email is being blocked but lack the resources to fix the problem.
The complaints come as the spam epidemic continues to fester, with unsolicited email comprising as much as 80 percent of all email, according to some studies. The price we pay in lost worker productivity and increased expenses for network providers are well documented. Less understood is the toll spam is taking on perfectly legitimate communications that go missing with nary a word. Microsoft managers say the spam threat requires they take drastic action and they are working to ensure their new techniques don't block benign messages.
El Reg has corresponded with two admins affected by SmartScreen and has run web searches - here, here and here - that suggest there are plenty others who are experiencing the same problem. In the cases we've checked, these servers are behind domains that are a year or more old, are correctly listed in the DNS and have solid SPF records - excluding the most common reasons a mail server might get blacklisted. They also run on a variety of platforms (including Sendmail and hMailServer), suggesting technical problems with a particular server are not the cause.
One of the admins is James Firth, founder of a UK-based IT consultancy, who began noticing emails sent from his company's relay were uniformly failing to be delivered to recipients with Hotmail accounts.
"They weren't going to a user's Junk mail box, nor were they being bounced," Firth says. "They were simply disappearing!"
So Firth began corresponding with Hotmail support people. After five days of back and forth, a Microsoft employee named Bobbi confirmed that emails sent from Firth's domain, daltonfirth.co.uk, were being "hard filtered" by SmartScreen. And not because they violated some documented technical requirement or contained suspicious phrases that triggered content filters. Rather, they failed to pass conditions buried deep inside SmartScreen that support people declined to share with Firth - out of concern the disclosures would allow spammers to bypass the defenses.
The support team suggested Firth confirm the emails complied to Hotmail technical standards. Firth checked, and they did. The support people also suggested Firth consider enrolling in a fee-based, third-party accreditation service called Sender Score Certified (price tag, according to a different source: $1,400 for the first year). He declined because Microsoft made no guarantees that doing so would solve his problems. More than a week after first discovering the problem, Firth still can't send emails to customers with Hotmail addresses.
"I simply can' believe that someone thought that this was a good idea, and think Hotmail users should be warned that Microsoft are choosing not to send their emails to them!" he says.
Microsoft representatives say the upsurge in spam - which they estimate to be about 40 per cent in the past year - has forced them to employ new techniques to safeguard Hotmail users and infrastructure. Once upon a time, messages were flagged largely based on the domain that sent it (e.g. Cyberpromotions.com) and by keywords - (think "get rich quick") that were embedded in them. With the growing use of image spam and botnets used to blast them to every corner of the globe, Microsoft has had to employ additional measures.
"The rules of deliverability have really changed," says Brian Holdsworth, senior product planner for Microsoft safety technologies, a group that is responsible for fending off the spam menace. "It's no longer just me, my email address and the domain I own. It's where I'm hosting it at and the reputation of my host."
Holdsworth was unfamiliar with Firth's case, but he speculates Firth's domain may be blocked because the provider hosting it may be home to a large number of infected machines. In such cases, all sites sitting on top of the host can be affected. Microsoft provides a service that allows hosts to review the reputation assigned to them by SmartScreen. (The service is not available to customers such as Firth.)
Microsoft representatives say they are making important progress in combating spam, having reduced the amount of junk mail landing in inboxes by half in the past year. What's more, of about 5bn messages sent to Hotmail users daily, as many as 4.5bn are spam. They say they are committed to resolving the problems such as Firth's and add that most large email providers are struggling with the same issues.
Perhaps, but according to Dominic Ryan, who runs a website called IIS-Aid, other providers do a better job. Ryan found that email his site tried to send to users with Hotmail addresses was also getting blocked, and he got to wondering how many other services might block his domain. So he set up a simple test in which he sent an identical email to accounts with ten different services.
Only one account - you can probably guess from which provider - failed to receive it.
"I have basically given up on Hotmail support," says Ryan, whose credentials include being a three-time Microsoft IIS MVP. He now bans the use of Hotmail addresses for users registering on his site and sends an automated response to those sending messages to fellow users with Hotmail accounts warning that the dispatches will probably not get through.
Of course, the biggest friendly-fire casualties in Microsoft's war on spam are the Hotmail users whose legitimate emails are killed before ever landing in their inbox. Firth and Ryan have been working diligently to alert these individuals to the problem, but so far they're warnings have gone unheeded. ®