Dailydave mailing list archives
Re: Debugging the false alarm problem.
From: dan () geer org
Date: Thu, 27 Sep 2007 15:57:57 -0400
Dave, et al., The answer to (almost) any testing problem is to do something that is multi-stage. The reason is that, as you were suggesting, it is impossible to eliminate all errors but -- and this is the good part -- you can bias one stage to err in a known direction and bias a subsequent stage to err in the opposite direction. This is what, at the hardware level, many signal filters do so as to get "band-pass" outputs, and it is likewise how we (they) can afford to screen all the blood supply for HIV. It works like this: Stage 1 misses nothing but has a high false positive rate. Because it misses nothing, that which it declares to be negative can be discarded as of no further interest. Stage 2 has the reverse characteristic or, as the more frequent alternate, Stage 2 is much better but much more expensive. In either case, you've used Stage 1 to rule-out and Stage 2 to rule-in. Not advertising, but this is in somewhat greater detail in slides 75-100 at http://geer.tinho.net/usenix/measuringsecurity.tutorialv2.pdf which also includes some of the standard terminology used by diagnostic testing and information retrieval folks, which terminology I suggest that we in the security field adopt. --dan _______________________________________________ Dailydave mailing list Dailydave () lists immunitysec com http://lists.immunitysec.com/mailman/listinfo/dailydave
Current thread:
- Debugging the false alarm problem. Dave Aitel (Sep 27)
- Re: Debugging the false alarm problem. dan (Sep 27)