Re: [Emerging-Sigs] Reliability of signatures

[Matt Olney <molney () sourcefire com>]

I'm pretty sure Matt meant "the (malware|exploit|policy violation|suspicious
traffic|etc) the rule is intended to trigger on.  It is only a false
positive if it alerts on something that isn't the intended target.  This can
happen for several reasons, some we can fix and some we can't.
 Unfortunately, it can be exceptionally difficult and time consuming to
learn what the rule is intended to detect on.  That information, especially
from the exploit detect side isn't necessarily publicly available.  This is
one of the reasons we like to get FP reports directly, so we can compare
research notes and ensure the rule is the best it can be.

That being said, we do use a lot of statistical analysis on which rules
alert the most, which rules are performance poor and which ones seem dead
on.  This can come from commercial customers, open source customers or from
research sensors we have deployed.

I think the idea of a community report on FPs is very useful.  I would think
that it would be useful for some organizations to have it hosted at
Sourcefire or another entity that can be trusted to secure pcap submissions
if the submitter wants to provide the data to a trusted source.  Maybe Joel
and Jonkman can work something out for information sharing.

Just some thoughts, back to Razorback dev.

Matt

On Thu, Feb 10, 2011 at 8:30 AM, Michael Stone <mstone+snort () mathom us>wrote:

> On Fri, Feb 04, 2011 at 02:01:05PM -0500, Matthew Jonkman wrote:
> > I agree on the difference between just logging hits and having true FP and
> TP ratings. But even a false positive can be different on the same packet in
> different organizations. Many folks mark a hit a false positive because it's
> just not of interest, vs nt hitting on what it's supposed to be looking for.
>
> Well, even that distincion isn't so clear. Does "what it's supposed to
> be looking for" mean "the string the signature was written against" or
> "the malware the signature was written against"?
>
> Mike Stone
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users