Re: Crossover Error Rate (WAS "Intrusion Prevention")

["Raistlin" <raistlin () gioco net>]

>  Just as with an IDS, you can reduce
> one at the expense of increasing the other, but unlike IDS, there's a
> commonly-known standard called the CER, or "Crossover Error Rate,"

That's not indicative, really.

In evaluating a system with that metric, you are supposing that both kind of
errors are equally costly. They could not be  (for example, in a biomedic
system it is FAR better to have a false alarm than a false negative !).

In addition it is not known, a priori, if the cost linearly scales. Having
10 false positives a day can be acceptable, 100 false positives may be a bit
more harassing (but not, necessarily, 10 times more), while of thousands of
false positives are completely unmanageable (they have an "infinite" cost:
we don't absolutely want to have that). At the same time, 1 false negative
may be bad,  and 100 false negatives are probably in the scale of "better to
launch this crap out of the window".

Please note that all the figures are totally subjective, and here only for
the sake of an example, do not flame me on the figures :P

What you really want to build is an ROC, Receiver Operating Curve, which is
a diagram with a measure of the false positives on X axis, and a measure of
the detection rate on the other. They are in some kind of 1/x - like
relationship (the more false positives you accept, the better you find
attacks, and vice versa). A "higher" graph (A larger area under it) means a
"better" system, on the whole. But more accurately, you can  match this
graph with your own "cost function" for false detections and misses, by
using really simple operational research techniques (you build the gradient
on the graph, and find the tangent with the ROC curve).

It's all theory with 40 years of background.

Stefano