Re: Announcement: Alert Verification for Snort

[Martin Roesch <roesch () sourcefire com>]

On Oct 23, 2003, at 8:31 PM, Sam f. Stover wrote:
> On Thursday, October 23, 2003, at 07:03  PM, Christopher Kruegel wrote:
>
> > From a theoretical point of view, I think that Marty is right and his 
> > classification is correct.
>
> I probably agree with you both "theoretically".  However, I was 
> talking about what actually happens to real users.  I used to work for 
> an IDS vendor, and I know how much of a glass bubble it can be.  Out 
> in the "real world" however, theory is vastly different than practice.

The "theory" here is that we need to get a better handle on how events 
generated by a NIDS actually apply to the network.  We're seeking to 
separate the things that can matter in your security environment from 
the things that can't.  I don't really see that as a glass bubble, I 
see it as a way to prove effective prioritization to the masses of data 
that IDS can produce.  If only 10 out of 10000 security events in your 
environment matter a day, it'd be nice to have something that points 
out what events they are.

In the quest to get to that point, I decided to start classifying the 
"false positives" that were not really false positives as a different 
class of detect, that's all.  When people start talking about all the 
"false positives" in Snort and other IDSes, it'd be nice if they 
differentiated the "screw up" false positives from the "need more data" 
false positives.

> >  In fact, we had a discussion about whether 'alert verification' was 
> > the correct term to use. We then concluded that most people don't 
> > care why they spent time looking at an alert that doesn't matter to 
> > them and that they refer to such alerts in general as false 
> > positives.
>
> This is *not* my experience.  I personally get extremely annoyed if 
> it's my fault (or the fault of the tool I chose to employ) that leads 
> me on a wild goose chase.  I want my IDS to learn with me, not 
> constantly provide me with the same level of annoyance.  It needs to 
> evolve.

I'd rather that the IDS behave deterministically based not only on what 
we told it to detect, but also on the effects of attacks that it sees 
and/or the vulnerability profile of devices on the network.  
Nondeterministic learning systems can go "crazy" if we're not very 
careful what we choose them to learn on...

--
Martin Roesch - Founder/CTO, Sourcefire Inc. - (410)290-1616
Sourcefire: Snort-based Enterprise Intrusion Detection Infrastructure
roesch () sourcefire com - http://www.sourcefire.com
Snort: Open Source Network IDS - http://www.snort.org


---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023 
and use priority code SF4.
---------------------------------------------------------------------------