Most security products not up to scratch

Only four per cent (one in 25) information security products pass muster when first tested under a widely-used industry kitemark scheme.

Most products require two or more cycles of testing before achieving certification, according to security testing outfit ICSA Labs.

ICSA Labs, which has tested anti-virus and firewall products for more than 20 years, reported in the study published Monday that problems included basic failures to perform often cropped up. Across seven product categories, core product functionality shortcomings accounted for 78 per cent of initial test failures.

For example, anti-virus products often failed to prevent malware infection at the first attempt, while firewalls or IPS (intrusion prevention) products failed to make the grade in blocking attack traffic.

Shortcomings in accurately logging data was the second most common reason for security product test failures, particularly for firewalls. Security flaws introduced by the products also created a frequent source of problems, along with "random behavior that affects product availability" (ie crashes).

The ICSA Labs testing and certification process is tough, with only four per cent of products tested making the grade during the first testing cycle. However, 82 per cent of products resubmitted for testing eventually pass and earn ICSA Labs certification, a benchmark they have to match periodically to gain ongoing certification.

Taken at face value the test failings might appear damning, but the test process itself helps to push vendors towards delivering more secure products. ICSA Labs details common pitfalls to avoid and factors that contribute to testing success in its report that vendors would do well to note.

Information security technology is even more subject to the vagaries of fashion than other aspects of IT, with the possible exception of mobile phone handsets.

ICSA Labs advises end users to choose simplicity over complexity, and suggests a bias towards more established products over newer products that whose kinks are yet to be worked out. The advice runs contrary to conventional industry marketing, which would have users believe innovation is making products better-performing and more secure.

ICSA Labs Product Assurance Report covers testing across seven product categories - anti-virus, network firewall, Web application firewall, network IPS, IPSec VPN, SSL VPN and custom testing. The study, which was sponsored by Verizon Business, can be found here (PDF). ®