Reg comments2

Detection systems guard against network intrusion

Prevention is better than cure

How do the different types of intrusion prevention system (IPS) work?

Inline systems sit on the network like layer-two bridges, passing traffic along as they receive it. Host-based systems sit on the server, watching the traffic that it sends and receives.

Both check packets for any suspicious activity, often using the most basic method of detection: signatures, or rules. If traffic passing through the network intrusion detection system matches a certain pattern, it can be blocked; traffic already identified as malicious doesn’t make it through.

Legible signatures

That traffic could fit a predefined suspicious pattern, such as aggressive port scanning, operating system fingerprinting, server message block probes, or even simple operations such as trying to login to telnet as root.

And, of course, emails with particular characteristics or strings can also be rolled into signature lists.

The biggest problem with signatures is that they are generally retroactive. They stop attacks that have already been documented, and are not so good at spotting those that have not yet appeared in the wild.

Statistical anomaly-based detection works differently. It begins by learning what normal behaviour on a network looks like, then uses it as a baseline to spot irregular activity.

This method has several advantages over signature-based detection. The first is that the system improves over time. The more traffic it is able to sample, the more knowledgeable it becomes about what constitutes acceptable activity.

The statistical anomaly method also does a better job of detecting zero-day attacks. Even though no signatures exist for these, the system often picks them up because they push traffic patterns outside what is considered “normal”.

Breaking and entering

On the downside, however, a statistical system doesn't understand the traffic it is analysing, and cannot make judgement calls about whether it is malicious or not. It simply looks for patterns outside those that regularly occur.

A wily attacker could introduce just enough of an attack pattern into the network to make the intrusion prevention system consider it normal. For example, small, non-aggressive port scans could be used to raise the system's tolerance.

There is a third method for detecting and preventing malicious traffic: stateful protocol analysis (aka deep packet inspection), which combines the best of both the other methods. It can be used to identify anomalies, so it has a better chance of detecting attacks without a particular signature.

Suspicious movements

It also understands the traffic it is analysing and can make more intelligent decisions about whether a pattern is suspicious, rather than simply detecting an anomaly and handing the whole tangled mess to an administrator to analyse.

These systems work by understanding what is expected during a particular application-layer session. For example, the system might know what response to expect back from a server during a particular authentication session. If that response does not arise (or a different one does) then it might raise an alert.

This approach can look for unexpected command sequences (such as a command issued to a server without a specific preceding command). It can also look for malformed protocols that might otherwise crash a system or cause a buffer overflow error.

The downside to this kind of analytical detection is that it often requires bigger computing resource.

Administrators need to tune devices to make them more effective

So, what kind of system should you buy? Ideally, you should have one that uses several approaches. The term “defence in depth” is more suited to the IPS product category than to anything else.

But however good it is, it won’t make your network watertight. NSS Labs, an organisation which tests security equipment without vendor funding, ran a group test of IPS products late last year. It found that out of the box, security effectiveness averaged about 62 per cent (with the lowest product scoring just 31 per cent). Performance decreased overall compared with a year previously.

Perhaps the most revealing result was that administrators need to tune the devices to make them more effective. When configured to an administrator's own preferences, average security efficiency rose to 83 per cent.

The moral? Don't expect these things to protect you entirely straight out of the box. Come to that, don't expect them to shield your backside even when you spend time helping them to understand your network better. A few nasties, it seems, will always get through. ®

Sponsored: The Joy and Pain of Buying IT - Have Your Say

Biting the hand that feeds IT © 1998–2017