Re: Announcement: Alert Verification for Snort

[Aaron Temin <aaron.temin () comcast net>]

Extending the categorizations (mentioned by Marty in his email and Ron
in the whitepaper referenced in his email) can lead, I think, to a
quantifiable description of the value of combining vulnerability
information with alerts from an IDS.

By (pedantically, I admit) separating "accurate" into positives and
negatives, the 9 categories in Ron's whitepaper turn into 16, with 4 of
those having different outcomes (that is, where the application of the
vulnerability knowledge modifies the outcome from the IDS itself).

Of these 4, 3 represent improvements and 1 represents a degradation. So,
assuming an equal distribution, combining the two sources of information
affects 25% of the outcomes, and improves the outcome 19% (3/16) of the
time.

Two of the 3 "improved" cases are where a false positive from the IDS is
downgraded/ignored, so this is one way to reduce false positives....

Two of the 4 are good nontextuals, where the vulnerability information
provides obviously useful additional information for determining if the
alert is important.

To get results that are more applicable to one's network, one could
provide a separate weighting factor to each possible outcome and then
combine those numbers (that is, depending on one's distribution of
correct and incorrect detects, the improvement could be larger or
smaller than 19%; one also has to consider how damaging the 1 degrading
case is to one's security- this is where the IDS indicates a true
detect, but the vulnerability information incorrectly says that the
systems are not vulnerable to the detected attack).

(I'm happy to provide an extended description of this if anyone is
interested, I was trying to keep this posting as short as possible.)

Aaron

On Thu, 2003-10-23 at 22:17, Ron Gula wrote:
> Good thread so far, but when you add in the fact that your vulnerability
> scanner can have false positives and false negatives, things get very
> complex pretty fast. I put a paper out on this earlier this year (see
> the papers section at www.tenablesecurity.com) and I broke the correlation
> out in nine areas. Both an IDS event and a Vulnerability Detect can have
> three states - false positive, false negative and being accurate. This
> actually gives you nine states to deal with.

<snip>


---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023 
and use priority code SF4.
---------------------------------------------------------------------------