RE: Crossover Error Rate (WAS "Intrusion Prevention")

["Rob Shein" <shoten () starpower net>]

> -----Original Message-----
> From: Raistlin [mailto:raistlin () gioco net] 
> Sent: Wednesday, December 11, 2002 2:16 PM
> To: focus-ids () securityfocus com
> Subject: Re: Crossover Error Rate (WAS "Intrusion Prevention")
>
>
> >  Just as with an IDS, you can reduce
> > one at the expense of increasing the other, but unlike IDS, 
> there's a 
> > commonly-known standard called the CER, or "Crossover Error Rate,"
>
> That's not indicative, really.
>
> In evaluating a system with that metric, you are supposing 
> that both kind of errors are equally costly. They could not 
> be  (for example, in a biomedic system it is FAR better to 
> have a false alarm than a false negative !).
 
Actually, that depends.  There are situations where a false accept is
worse than a false reject, and vice versa.  The point of the CER is
merely to keep vendors from being able to cook the figures by tuning
systems unrealistically for tests.  (See under "IDS vendors who claim
zero false positives.")  In the end, however, it has proven true that
the lower the CER, the more accurate and reliable the biometric system
is, regardless of the specifics.  And my hope is that a similar method
can be developed for network-based IDS...it won't be a magic bullet for
selection, but it would definitely clear some of the fog so that people
who have to evaluate technologies can focus on their specific needs more
than sorting out the truth from the half-truth.