Feeds

AV-Test boss dismisses Microsoft criticism of malware test results

Says Redmond's not picking up the fresh threats

The Essential Guide to IT Transformation

A war of words has broken out over security testing, with Microsoft and the AV-Test Institute going head-to-head over Redmond's failure to qualify for the last round of certification from the German testers.

On Tuesday, AV-Test announced its December round of security software evaluations, and both Microsoft's Security Essentials and its Forefront suite failed to achieve certification for the second month running. The next day, Microsoft questioned the results in an extended blog post by Joe Blackbird, program manager at Microsoft's Malware Protection Center.

"Our review showed that 0.0033 percent of our Microsoft Security Essentials and Microsoft Forefront Endpoint Protection customers were impacted by malware samples not detected during the test," Blackbird said. "In addition, 94 percent of the malware samples not detected during the test didn't impact our customers."

But Andreas Marx, CEO of AV-Test, told El Reg that one would expect this. "I'm not surprised that in testing two months later they get different results," he said.

The AV-Test results show that Microsoft's twin security programs protected against 100 per cent of known threats, as did every other security suite. The two packages produce low rates of false positives in comparison to the competition and are significantly lighter on processor load during operations.

But where Redmond is falling down is in protecting against zero-day attacks. Security Essentials and Forefront both scored last in this regard among all the suites tested, getting 78 per cent of zero-days apiece. Blackbird said that AV-Test attached too much importance to the zero-day threat in its metrics, since that section of the testing accounts for 50 per cent of the final score, but Marx argued that zero-day performance was crucial to real-world threats.

He explained that many security tests rely on the "wild list" of malware in circulation among the community, but that the samples can be 80-90 days old in some cases and will therefore have been supplanted by newer varieties of core malware routines. As such, testing against current threats is key to the institute's metric.

AV-Test teams take malware that is minutes old, Marx explained, and run the data into the security testing suite. A testing process carried out by Microsoft much later would be bound to cover the malware tested, since samples would already have been reported.

"Today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," Marx explained.

"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high."

Overall AV-Tests did show that the effectiveness of antivirus code has declined. With this latest batch of tests, the average zero-day malware blocking success rate was 92 per cent. That doesn't sound too bad, but given the colossal amount of crooked code out there, it still represents a big problem – not to mention the ever-present threat of completely unknown families of malware coming online.

BitDefender, Kaspersky, and Norton topped December's consumer security rankings from AV-Test, with Kaspersky, F-Secure, and Sophos getting the best scores for business protection. ®

Build a business case: developing custom apps

More from The Register

next story
14 antivirus apps found to have security problems
Vendors just don't care, says researcher, after finding basic boo-boos in security software
Only '3% of web servers in top corps' fully fixed after Heartbleed snafu
Just slapping a patched OpenSSL on a machine ain't going to cut it, we're told
How long is too long to wait for a security fix?
Synology finally patches OpenSSL bugs in Trevor's NAS
Israel's Iron Dome missile tech stolen by Chinese hackers
Corporate raiders Comment Crew fingered for attacks
Roll out the welcome mat to hackers and crackers
Security chap pens guide to bug bounty programs that won't fail like Yahoo!'s
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
Researcher sat on critical IE bugs for THREE YEARS
VUPEN waited for Pwn2Own cash while IE's sandbox leaked
Four fake Google haxbots hit YOUR WEBSITE every day
Goog the perfect ruse to slip into SEO orfice
Secure microkernel that uses maths to be 'bug free' goes open source
Hacker-repelling, drone-protecting code will soon be yours to tweak as you see fit
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Build a business case: developing custom apps
Learn how to maximize the value of custom applications by accelerating and simplifying their development.