Re: [Emerging-Sigs] Reliability of signatures

[Jacob Kitchel <jacob.kitchel () gmail com>]

On Thu, Feb 10, 2011 at 9:17 AM, Matthew Jonkman
<jonkman () emergingthreatspro com> wrote:
> I agree with Matt here completely, (see previous thread about the greatness
> of Matts).
> But we both have the point of view of signature writers, whereas an
> analyst/incident responder in the field has a very different idea of an FP.

That may be true, that every person in the field has a different idea
of what an FP is. However, it doesn't mean they're all right. There is
one definition of a False Positive. As complicated as our environments
are we can't afford to use multiple definitions of the same concept
because it leads to confusion and lack of specificity when describing
a problem (among other things).

False Positive = when a sig fires on network traffic that is not what
it was intended to fire upon
"Does not apply to this host" or lowered severity = when a valid sig
fires on network traffic as the sig writer intended it to but the
network traffic is going toward a destination which is not running the
affected software/service/whatever.

If the viewing/management mechanism used to review the alert record
has the ability to evaluate and mark severity, that's a good thing.
But if it doesn't, then it has to exist in the analyst's brain and the
analyst has to make the right decision.

-Jacob

> So, just like there are 2 philosophies in how to tune a ruleset (i.e. only
> fire on compromises vs I want to know who's making attempts and block), I
> think we have to just accept that there are many definitions of a false
> positive, and they're all valid.
> And thus a "Report this as an FP" button is wildly complicated....
> Matt
> On Feb 10, 2011, at 10:04 AM, Matt Olney wrote:
>
> And I would argue that "no iis" here isn't a valid FP.  The signature
> performed correctly and notified you that a scan attempt was under way.  It
> is up to the system admin to correctly suppress/disable/modify rules that do
> not target his network.  In our view, a FP only occurs when network traffic
> triggers an alert that is specifically NOT traffic that the rule was
> intended to fire on.  The rules are application/server agnostic (some wiggle
> room in this comment both currently and in the future) they are solely based
> on the traffic on the wire.
>
> On Thu, Feb 10, 2011 at 10:00 AM, Michael Scheidell
> <michael.scheidell () secnap com> wrote:
> > On 2/10/11 9:55 AM, Matt Olney wrote:
> >
> >  Also, SPAM isn't an IDS issue, at
> >
> > ah, maybe I should have explained.  you missed the point.
> >
> > there needs to be a definition of FP vs MP for signatures for any of this
> > to mean anything.
> > one mans SPAM is another mans HAM,  one mans FP is another mans legit hit.
> > would you BLOCK on FP #2? maybe not inline, but I sure would blacklist the
> > ip.
> >
> > maybe in the 'human verified checkbox' you give them the ability to mark:
> >
> > [ ] FP:  signature too broad. matched legit traffic
> > [ ] FP:  no IIS servers here,
> >
> > (just so we have something for them to check)
> >
> >
> > --
> > Michael Scheidell, CTO
> > o: 561-999-5000
> > d: 561-948-2259
> > ISN: 1259*1300
> > > | SECNAP Network Security Corporation
> >
> > Certified SNORT Integrator
> > 2008-9 Hot Company Award Winner, World Executive Alliance
> > Five-Star Partner Program 2009, VARBusiness
> > Best in Email Security,2010: Network Products Guide
> > King of Spam Filters, SC Magazine 2008
> >
> > ________________________________
> >
> > This email has been scanned and certified safe by SpammerTrap®.
> > For Information please see http://www.secnap.com/products/spammertrap/
> >
> > ________________________________
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb_______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> ----------------------------------------------------
> Matthew Jonkman
> Emergingthreats.net
> Emerging Threats Pro
> Open Information Security Foundation (OISF)
> Phone 765-807-8630
> Fax 312-264-0205
> http://www.emergingthreatspro.com
> http://www.openinfosecfoundation.org
> ----------------------------------------------------
>
> PGP: http://www.jonkmans.com/mattjonkman.asc
>
>
>
>
> _______________________________________________
> Emerging-sigs mailing list
> Emerging-sigs () emergingthreats net
> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>
> Support Emerging Threats! Subscribe to Emerging Threats Pro
> http://www.emergingthreatspro.com
> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
> Current!

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users