Feeds

Anti-spam packages ‘too unreliable’ to certify

Testing. Very Testing

  • alert
  • submit to reddit

Choosing a cloud hosting partner with confidence

Poor performance of anti-spam packages is frustrating attempts by a leading certification vendor to develop benchmarking standards.

ICSA Labs, which runs one of the most important security industry certifications programmes, has recorded disappointing results in its preliminary tests of eight open source and commercial anti-spam packages.

Larry Bridwell, programme manager at ICSA Labs, an independent division within TruSecure, said it had "trouble getting up to 60 per cent" recognition of spam email in the products it has tested thus far.

ICSA Labs is well known for testing a wide variety of security products - anti-virus, cryptography, PKI, IPSec, VPN, firewall, intrusion detection and content filtering - against specific criteria for compliance and reliability. Products which pass the test - and most do not at the first attempt - earn the right to bear the ICSA Labs Certification Seal. Further testing is carried out to make sure products continue to meet certification standards.

Plans to create a certification programme for anti-spam products flowed out of ICSA's work in certifying content filtering packages.

George Japak, VP of ICSA Labs, told us that in the "near term there's unlikely to be a formal certification programme" for anti-spam products.

"It's hard to say 60 to 70 per cent is good enough," he said.

However stakeholders in ICSA certification programme - including developers, corporate end-users and independent researchers - are arguing that even 60 per cent efficiency contributes to reducing the spam problem.

In our own use of anti-spam packages like SpamPal and MailWasher, we found it fairly straightforward to achieve (we estimate) efficiency levels for blocking spam of 80 per cent or above. However this has at the expense of occasional (less than one in 100) false positives.

There is a disadvantage in using these products of mail downloads slowing to a crawl but the overall benefits of these packages outweigh their pitfalls. We also hear good things about SpamAssassin.

Doubtless other products and services achieve better results but without comprehensive independent testing it's hard to be sure what package to use.

The recent proliferation of anti-spam products and difficulties in agreeing testing criteria are complicating factors.. In testing only eight products, ICSA Labs is looking at only a very small sample of the scores of packages out there - a point that that hasn't gone unnoticed at ICSA.

We shouldn't read too much into ICSA Labs preliminary results, but they do point to the immaturity of the anti-spam market.

Longer term the future looks much brighter. With the best minds in the IETF, and elsewhere, focusing on the problem, gives reason for optimism.

Current research efforts to apply Bayesian analysis in weeding out spam messages from legitimate email and to slow spammers down using tar pits appear most promising. ®

Related stories

On spam cures that are worse than the disease
Web giants to declare war on spam
Florida spammers sue anti-spam groups
Rise of the Spam Zombies
IETF aims to can spam
Spammers break law with covert tracking
Plaid up in arms as Commons spam filter bans Welsh
Vendors complete tougher ICSA 4.0 firewall tests

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
You really need to do some tech support for Aunty Agnes
Free anti-virus software, expires, stops updating and p0wns the world
Regin: The super-spyware the security industry has been silent about
NSA fingered as likely source of complex malware family
You stupid BRICK! PCs running Avast AV can't handle Windows fixes
Fix issued, fingers pointed, forums in flames
Privacy bods offer GOV SPY VICTIMS a FREE SPYWARE SNIFFER
Looks for gov malware that evades most antivirus
Patch NOW! Microsoft slings emergency bug fix at Windows admins
Vulnerability promotes lusers to domain overlords ... oops
HACKERS can DELETE SURVEILLANCE DVRS remotely – report
Hikvision devices wide open to hacking, claim securobods
prev story

Whitepapers

Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Designing and building an open ITOA architecture
Learn about a new IT data taxonomy defined by the four data sources of IT visibility: wire, machine, agent, and synthetic data sets.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?