Feeds

Windows Defender spyware-blocking under fire (again)

Who tests the testers?

Intelligent flash storage arrays

Microsoft's Windows Defender has once again come under criticism for alleged shortcomings in blocking invasive spyware applications.

In tests sponsored by anti-spyware vendor PC Tools, and carried out by independent testing facility Enex Testlabs, Microsoft’s Windows Defender blocked less than half (46 per cent) of current spyware threats, scoring well below third party anti-spyware providers.

The findings, published on Tuesday, follow earlier in-house research by security rival Webroot that Windows Defender failed to block 84 per cent of a testing sample-set that included "15 of the most common variations of existing spyware and malware". Threats of various types - including adware, system monitors, key loggers and Trojans - were able to reside on the testing environment undetected by Windows Vista, Webroot reports.

Microsoft has declined to comment on Webroot's criticism and is also staying quiet on the latest reports.

Microsoft's rivals are understandably keen to promote the message that while Vista might be more secure than previous versions of Windows, users still need additional protection from malware threats. Redmond itself isn't up to the job so users ought to continue relying (buying) third-party products, the argument goes.

The problem with both the PC Tools and Webroot's survey is the result of the tests depends on the spyware sample used, who supplies it and the complete objectivity of the testing agency. In the case of the Webroot test, the sample data was "randomly chosen from a database of over 8,000 spyware installation programs that was provided by Webroot" (our emphasis). PC Tools criticised this approach as "hand picking" the sample set. It said PC Tools did not choose or supply the sample-set used by Enex, described as real-world spyware threats circulating in 2006.

According to the aggregate Enex test results for the whole of 2006, Microsoft’s Windows Defender quick scan was able to block only 47 per cent of dangerous threats while their full scan blocked 53 per cent. Tested at the same time and using the same sample-set, PC Tools’ Spyware Doctor quick scan blocked 83 per cent and the full scan blocked 89 per cent better than other unnamed anti-spyware products put through their paces.

However since PC Tools hired Enex to conduct the tests they inevitably carry less authority than would be the case if they were done completely independently.

In the anti-virus world, vendors have agreed to submit to testing against a set of viruses at large on the internet in tests conducted by independent testing houses such as Virus Bulletin and not paid for by any one vendor. The anti-spyware industry hasn't reached this level of maturity.

It's quite possible, for example, that Webroot and PC Tools products might only detect a small proportion of a sample set supplied by Microsoft or McAfee or anyone else. That's not to say these products are ineffective, simply that tests are meaningless until an independently-produced sample is used in tests conducted by wholly disinterested parties.

Until we get to that point anti-spyware tests will be more about marketing than objective product performance assessments. ®

Top 5 reasons to deploy VMware with Tegile

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.