AVG scanner blasts internet with fake traffic
You're not that popular after all
Exclusive Early last month, webmasters here at The Reg noticed an unexpected spike in our site traffic. Suddenly, we had far more readers than ever before, and they were reading at a record clip. Visits actually doubled on certain landing pages, and more than a few ho-hum stories attracted an audience worthy of a Pulitzer Prize winner. Or so it seemed.
As it turns out, much of this traffic was driven by the new malware scanner from AVG Technologies.
Six months ago, AVG acquired Exploit Prevention Labs and its LinkScanner, a tool that automatically scans search engine results before you click on them. If you search Google, for instance, and ten results turn up, it visits all ten links to ensure they're malware free.
Then, in February, AVG paired LinkScanner with its anti-virus engine, which has about 70 million active users worldwide. The company estimates that 20 million machines have upgraded to its new security suite, AVG version 8, and this has already cooked up enough ghost clicks to skew traffic not only on The Reg but any number of other sites as well.
Adam Beale, who runs a UK-based internet consultancy, says that across his small stable of clients, traffic has spiked as much as 80 per cent on some sites. And this is more than just an inconvenience. After all, sites live and die by their traffic numbers. And net resources aren't free.
"Although [the AVG LinkScanner] might be good for the security of users, it's a real pain for website owners and webmasters," Beale tells us, having blogged about this growing problem. "It's causing people to think their traffic is increasing, costing those who pay for bandwidth, and wasting disk space with large amounts of unnecessary lines in log files."
One of his clients, Beale says, normally pulls in 140GB of bandwidth a month, and for June, he predicts a 5 per cent jump.
When we spoke to AVG chief of research Roger Thompson earlier this week, he was unaware of these issues. But he defended the role of LinkScanner, which he designed while serving as CTO of Exploit Prevention Labs.
"There's so much hacking activity going on the web. The only way to really tell what's there is to go and have a look," he told us. "I don't want to sound flip about this, but if you want to make omelettes, you have to break some eggs."
But what about webmasters?
Webmasters deal with robot traffic and other rogue visits all the time. But this is a little different. In an effort to fool even the sneakiest malware exploits, LinkScanner does its best to imitate real user clicks - which means most webmasters are completely unaware of the problem.
At the moment, there is a way of filtering AVG traffic from log files. But it's unclear whether this method would bag legitimate traffic as well. And Thompson suggests that - in the name of high security - AVG may make changes that prevent such filtering.
That could destroy web analytics as we know it.
"A situation like this where there is in effect false traffic, where something is generating what is bogus data, leads to wrong budget decisions and marketing activities," says Barry Parshall, director of product management at WebTrends, a popular web analytics firm. "I completely get the value proposition [of LinkScanner], but it would be responsible of them to identify themselves, with agent code or whatever it might be, so legitimate businesses can serve their customers properly."
I quite agree with those of you who wrote that AVG was now too bloated, had too many false detections, has been irresponsibly released with this terrible link scanning tech.
I've used AVG AV for years but sadly these changes in version 8 are just unacceptible. I'd go without AV protection at all before I'd run AVG8.
@ John A Thomson - We didn't need to "learn" what you had to say, it was obvious and thoroughly weighed by others who had enough sense to see the problems with linkscanner far outweigh the dubious benefits.
The obvious answer has already been mentioned, get rid of linkscanner and use a proxy if it's really that important. The issue of infection method over file identification is not relevant, that can be detected after the link was clicked and content cached locally.
A 2000% increase in bandwidth use!
Having been trying to discover the source of high server loads and spiraling bandwidth use since 24th May, I finally tracked the issue down to this AVG scanner - it has caused a 2000% increase in daily traffic from my server on a reasonably small site. The site usually accounts for just 14GB/month but so far in June we're up to 300GB.
Even worse, the requests aren't to real pages and are all generating 404 errors - literally hundreds of thousands of them. I have had to turn off a custom 404 error page because of this to reduce what my Apache server has to do.
Fantastic. I didn't like AVG before this, but now I am going to actively tell my clients never to use it.
Re: Response from AVG
Well how about mailing back to you the report of bad sites and you can collect them and inform the website owner?
How about scanning as you download, rather than scan-ahead?