Better metrics needed for security, says expert
Awash in bad data
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
BOSTON — The security industry has done a poor job of finding ways for companies to measure their security, but that does not mean that collecting data is not valuable, the former head of the U.S. Department of Homeland Security's cyber group told attendees at the SOURCE Boston conference on Thursday.
Amit Yoran, CEO of security firm NetWitness and the former director of the National Cyber Security Directorate at the DHS, criticized today's risk management practices. The security industry is awash in bad data, and companies that attempt to use the metrics could take the wrong actions, he said.
"When you are trying to boil a very complex topic into ... a discrete number to management, you end up driving organizational behavior toward bad metrics," Yoran said.
Yet, foregoing data collection is not the right path, either.
"Rather than say, 'Don't measure any data, because it will backfire,' we need to measure everything," he said. "Once you have the metrics, you can analyze the data in different ways," and find those that actually help the company minimize risk.
The process requires that executives work with their security group to find the right way to measure security for that specific company, he said.
"The security culture has to be set by the executives," Yoran said. "Set the expectations that a lack of due care is not going to be tolerated."
This article originally appeared in SecurityFocus.
Copyright © 2009, SecurityFocus
COMMENTS
Re: need to measure everything
But isn't the real problem that what you are actually trying to measure is what you are NOT doing, so that you can identify it and do something about it. Of course if you could measure it, then you would know about it, and knowing about it you would have done something about it.
Or not, depending on your local implementation of the PHB.
Gosh
"[...] Former head of the U.S. Department of Homeland Security's cyber group [...]: 'we need to measure everything'"
Why am I not surprised?
The problem with risk management
Fundamentally, the whole metrics racket is a hiding to nothing when it comes to security. Suppose you have a webserver, and for the ease of the example, let's say your company does all it's business through this system. It has a bug. Should you patch it? Well, you can put a number on amount of business lost through downtime when patching (yes yes, no clustering here, we're in Gedankensville, OK?) You might even people to put a wet finger in the air and come up with some sort of guesstimate about how much it would cost you in lost business and reputational damage should you arrive at work one day to find your front page replaced by Fleshbot (although I'd argue that value is entirely theoretical, and anyway is probably a hell of a lot less than you might like to think it is - especially if you're the person trying to diddle the numbers so you get a bigger budget next year.) One thing you absolutely /cannot/ do, though, is put any sort of probability value on the chances of getting pwned /through that specific vulnerability/, per day. That's the sort of numbers the insurance business like to crunch to work out your car insurance premium, and why middle-aged me pays less to insure my 250 BHP turbo-nutter-bastard mobile than a 22 yo with a hot hatch, set of alloy wheels, Haynes manual and a bodykit :) )
Given that the final number at the bottom of the page that's supposed to allow you to rank your systems, the stuff you could do to secure them, and how much you should spend to do so are based on garbage - and we've all seen what lousy risk management based on garbage input can do to the world economy - it follows that one should beware of snake-oil salesmen bearing metrics.
Mine's the one with the "kick me" note stuck on the back by the Sales Director...
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
