Time to blacklist blacklists
Plenty of ways to overcome 'minor inconvenience'
Blacklists have their place for detecting and identifying malicious content and activity, with the whole signature-based malware detection industry effectively being built around the concept that blacklists are reliable mechanisms.
The only problem is that they aren't.
They certainly are an important element of security models, but the last couple of decades of security research has shown that they quickly become ineffective in the face of a rapidly evolving threat.
Early in the life of antivirus tools, simple signature based detection was enough. An internal blacklist could identify all known pieces of malware because they did not evolve or spread very rapidly. When polymorphic malware began to exhibit better software development, the need for heuristic detection engines became more urgent. Most antimalware software now has a combination of blacklisting and heuristics in use to assist in identifying malicious activity (when they aren't busy deleting critical system files or being compromised by their own analysis engines).
Having an exhaustive blacklist helps companies claim that they detect many tens of thousands of viruses and malware, when in reality it may be many different versions of a few key pieces of malware, just different enough from previous versions to require a brand new blacklist signature.
Moving on to blacklists of known spam-generating IPs and malware-serving sites, we start to see significant problems emerge with this particular approach to protection.
Many mail server administrators will have encountered at least one period where they have found their IP on an RBL (Real Time Block List) alongside IPs that have seen to be spewing spam across networks (or they could have just had AOL mailing list subscribers who find it easier to report as spam than unsubscribe from something they manually subscribed to). With the use of dynamic IP addresses and virtual hosts, many have found that if they have a bad network neighbour, they can be hit with the same blocking (we've had it happen a few times) from indiscriminate RBL maintainers.
Even important registries are not immune from arbitrary blockage and ongoing annoyance from poorly developed RBLs.
The problem of misidentification becomes even worse when blacklists of websites that are hosting malware and phishing attacks are maintained. Microsoft, Mozilla, Opera, McAfee, and Google are just some of the large bodies that have invested significant resources to the creation, maintenance, and use of website blacklists to warn users of potential malicious activity on websites (and in some cases prevent access).
Anyone who spends even just a little bit of time involved with researching and observing the patterns and pace of website attacks, hacks and defacements will know that websites are essentially fragile entities and it doesn't take much for a well-trusted site to become a malware-spewing nightmare.
Like trying to use DRM to restrict the spread of copyright infringement, using blacklists / blocklists to limit access to sites will only stop the honest, and the casual attacker (extremely casual attacker) from getting people to see their site. Any attacker that is remotely serious about their work will have plenty of ways to bypass and overcome the minor inconvenience that the blacklists pose.
If any further evidence was required, a security researcher (Kuza) has published a small set of techniques that can be used to bypass these website blacklists. The set of techniques published reflects just a small number of the many different ways that it is possible to avoid these lists, not least of which is the fact that it takes time for a site to be added to a blacklist.
The response that Kuza received from Microsoft when he reported his techniques for phishing detection avoidance is actually quite an intelligent response - "[it] is not a security feature".
The only problem with this is that many, many people (including a lot of 'security' people who should really know better) consider these lists to be just that - a security feature.
It is time that people became aware that these lists are a small tool of their protection arsenal, and not the major innovation that their creators and maintainers describe them as. It is also time that people became aware of the problems that these lists can cause when improperly developed and maintained (and even when they aren't).
This article originally appeared at Sūnnet Beskerming
© 2007 Sûnnet Beskerming Pty. Ltd
Sūnnet Beskerming is an independent Information Security firm operating from the antipodes. Specialising in the gap between threat emergence and vendor response, Sūnnet Beskerming provides global reach with a local touch.
Yes, I reject, not bounce !
To Andy, please go back and read what I wrote - I do NOT bounce messages since, as you say, it would be cretinous. Actually that is doing a disservice to cretins to associate them with such assinine behaviour.
As Sebastian correctly points out, if you REJECT the message during the initial SMTP dialog then in the vast majority of cases the message just gets dropped as the smap software the user is infected with doesn't bother with failures. The ONLY situation when such rejections result in backscatter (the sending of bounce messages to innocent third parties) is if the message was relayed through a properly functioning mail server - but that is rare and spam software doesn't normally try and do that because they know that most of their messages would never leave the building if they did.
But, in the case of false positives, the user WILL get a bounce message and know that their mail didn't get through - which I think is very important. I really cannot think of anything more stupid than accepting mail and then silently deleting it, which is of couse what most people do. Lets face it, there's a whole industry selling solutions based on just silently deleting email ! In fact, I'm currently having 'discussions' with my ISP who won't let me NOT use their server as a backup MX*, and it is porgrammed to do just that - apart from the fact that it's broken.
* And I can't move my DNS elsewhere (like home) because if I do then their web hosting turns off for the domain.
Re: Don't bounce spam
There's two kinds of rejecting mail which need to be distinguished here:
- Accepting all mail first, and then sending a bounce when messages proves undesirable/undeliverable. This is indeed bad, as in the case of spam the bounce will go to a spoofed address.
- Refusing the message during the SMTP dialogue. This way, the receiving server never becomes responsible for delivery of the message, rather the sending server has to send the bounce. Ideally, unwanted messages will never leave their origin this way, rendering spoofed sender addressed ineffective.
So I hope Simon is using the second kind of rejecting.
Don't bounce spam
Simon Hobson: "I reject rather than discard mails that fail the checks so the user knows that their message has been blocked."
That is cretinous.
As *everyone* knows, pammers systematically falsify the sender address, so whoever gets notified, it certainly isn't the sender. Unless it was a false positive. But you wouldn't use a filter that generated false positives, would you?
Bouncing spam is exactly the same as relaying it. Don't do it.