Auto vulnerability scanners turn up mostly false positives
And they're still cheaper than looking for attacks manually
Nullcon Automated vulnerability scanners turn up mostly false positives, but even the wild goose chase that results can be cheaper for businesses than manual processes, according to NCC Group security engineer Clint Gibler.
At the Nullcon security conference in Goa, India, Gibler said he pointed an unnamed automated scanner at 100 of NCC's customers across 10 industry sectors.
The result of that effort was some 900,000 security-related red flags, and a false positive rate of 89 per cent in some industries. Even the scanner's "best" result produced around 50 per cent false positives.
The scans were conducted between February 2014 and May last year, scanning each company four times with all results manually vetted by NCC Group staff.
Gibler told Nullcon that he estimated the resources used chasing false positives is huge, but still says automated scanners are worth it for most companies.
His assertion is based on a security engineer being paid a wage of US$75,000, and taking less than a minute to assess each flaw discovered by an automated scanner.
“The amount of time people would waste vetting these false positives ranges between one and nine weeks which is a huge amount of time,” Gibler says.
“In the best case you're spending US$1000 in staff time to vet these issues (including true positives) and in the worse case up to US$10,000 to US$16,000.
“Most people when purchasing tools look at the price but there are these hidden factors that people don't consider about how long it takes to vet those results and how many are actually useful.”
However, Gibler told Vulture South automated scanning tools are valuable because they help bridge the gap between expensive penetration tests.
“I think they are still valuable useful and will become more so in the future,” he says.
Gibler informed customers of the discovered flaws but it still took most between 10 to 20 weeks to patch, and some dragged their feet leaving holes unfixed for a year.
The 10,000 discovered cross-site scripting vulnerabilities were the largest class of vulnerability NCC's tests found, among the 9000 different security flaws uncovered in the scans.
The majority of results affected companies in the leisure and media sector with 25,769 results and public sector education with 15,550.
Gibler said companies did not necessarily fix high-severity flaws faster than low risk bugs. ®