We previously discussed data-centric defenses as the critical last line of defense. As such, one of our requirements is to try and make it as airtight as possible. That’s not a minor or trivial requirement. In this article, we’ll discuss how we can get there.

There are two concepts we’ll need to discuss:

• False Positives and False Negatives
• IPS and IDS

False negatives

False Negatives or type 2 errors, are when a security system fails to react to an attack. So, our objective is to reduce or try to eliminate false negatives. The better we do that, the more airtight our security.

False negatives are the dirty little secret of security systems – the subject no one talks about. No one talks about how many false negatives they have, how to measure them, estimate them, and generally, about the effectiveness of security. It’s an important subject, but one that everyone tries to avoid.

To understand how to reduce false negatives, we must first understand false positives.

False positives

A false positive or type 1 error, is the opposite – when a security system incorrectly classifies legitimate activity as an attack. For example, when a good email gets into the spam folder.

False positives can be annoying, as in the case of misclassified emails. But they can also prevent people from doing their jobs when, for example, they can’t log in to a system they need for work. These are examples of false positives in preventive security. Because they can be debilitating and cause a lot of complaints, IPS is designed and tuned to have low or, ideally, no false positives.

The Balance

As you can imagine, there’s a trade-off between false positives and false negatives. Reducing one tends to increase the other. So, as we reduce false positives so people can do their jobs, we inevitably increase false negatives, and more attacks go undetected.

Another way to look at it is the sensitivity of the security system. A sensitive system will detect more attacks but have a lot of false alerts. A system calibrated to be less sensitive will have fewer false alerts but miss many attacks.

IPS & IDS

That brings up the other subject of IPS and IDS. There’s a common misconception that prevention is more important than detection. The logic behind it is, why detect something when you can prevent it? It sounds like a good idea, but it’s wrong.

As we just said, in IPS, we have to reduce false positives, and that increases false negatives. But IDS isn’t required to have zero false positives. In IDS, we even expect some level of false positives. False positives in IDS are false alerts that security people receive. They can be annoying if they’re too frequent, but we expect them to a certain degree. Being able to accommodate some false positives lets us significantly reduce the false negatives. In other words, we calibrate IDS systems to be more sensitive and detect a much higher number of attacks.

The diagram below shows the ideal calibration of IDS and IPS with no false positives for the preventive and no false negatives for the detective. The area between Preventive and Detective illustrates in Red the attacks that go undetected by the preventive (its false negatives) and in Blue the false alerts of the detective (its false positives).

If there were false positives for the preventive, that would show as Blue crossing right over the preventive line and getting blocked. Similarly, false negatives for the detective would be Red crossing left over the detective line and not being detected.

Using a combination of IDS and IPS is a simple way to estimate the number of false negatives in the IPS. There are other ways of doing that using static analysis.

So what’s better, IDS or IPS?

The truth is that we need both. Preventive to block as much as possible and detective to identify and alert about the rest. That’s how we get close to airtight.

There are other strategic mechanisms for getting close to airtight, like using serial and partially overlapping defenses. That takes us to the risk-control matrix, which is the subject of another article.

If you want to know more or have a free discussion with one of our experts, please contact us at info@bluecoreresearch.com