Re: [Emerging-Sigs] Reliability of signatures

[Matthew Jonkman <jonkman () emergingthreatspro com>]

I agree with Matt here completely, (see previous thread about the greatness of Matts).

But we both have the point of view of signature writers, whereas an analyst/incident responder in the field has a very 
different idea of an FP. 

So, just like there are 2 philosophies in how to tune a ruleset (i.e. only fire on compromises vs I want to know who's 
making attempts and block), I think we have to just accept that there are many definitions of a false positive, and 
they're all valid. 

And thus a "Report this as an FP" button is wildly complicated....

Matt

On Feb 10, 2011, at 10:04 AM, Matt Olney wrote:

> And I would argue that "no iis" here isn't a valid FP.  The signature performed correctly and notified you that a 
> scan attempt was under way.  It is up to the system admin to correctly suppress/disable/modify rules that do not 
> target his network.  In our view, a FP only occurs when network traffic triggers an alert that is specifically NOT 
> traffic that the rule was intended to fire on.  The rules are application/server agnostic (some wiggle room in this 
> comment both currently and in the future) they are solely based on the traffic on the wire.
>
> On Thu, Feb 10, 2011 at 10:00 AM, Michael Scheidell <michael.scheidell () secnap com> wrote:
>
>
> On 2/10/11 9:55 AM, Matt Olney wrote:
> >  Also, SPAM isn't an IDS issue, at
> ah, maybe I should have explained.  you missed the point.
>
> there needs to be a definition of FP vs MP for signatures for any of this to mean anything.
> one mans SPAM is another mans HAM,  one mans FP is another mans legit hit.
> would you BLOCK on FP #2? maybe not inline, but I sure would blacklist the ip.
>
> maybe in the 'human verified checkbox' you give them the ability to mark:
>
> [ ] FP:  signature too broad. matched legit traffic
> [ ] FP:  no IIS servers here, 
>
> (just so we have something for them to check)
>
>
>
> -- 
> Michael Scheidell, CTO
> o: 561-999-5000
> d: 561-948-2259
> ISN: 1259*1300
> > | SECNAP Network Security Corporation
> Certified SNORT Integrator
> 2008-9 Hot Company Award Winner, World Executive Alliance
> Five-Star Partner Program 2009, VARBusiness
> Best in Email Security,2010: Network Products Guide
> King of Spam Filters, SC Magazine 2008
>
> This email has been scanned and certified safe by SpammerTrap®. 
> For Information please see http://www.secnap.com/products/spammertrap/
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb_______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users


----------------------------------------------------
Matthew Jonkman
Emergingthreats.net
Emerging Threats Pro
Open Information Security Foundation (OISF)
Phone 765-807-8630
Fax 312-264-0205
http://www.emergingthreatspro.com
http://www.openinfosecfoundation.org
----------------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users