RE: False Positives

["Dudley, Brian (ISS Chicago)" <BDudley () iss net>]

While Blade Software's IDS informer is not actually compromising the host, it is generating legit attack traffic and 
thus the definition of it generating *false positives* is not entirely accurate.
 
As Steve Richards stated, there is a big difference between a false positive (a bug in the product) and a non-security 
event (an alert that has no impact on the organization based on the configurations, mitigating controls or security 
policy).
 
"Any IDS system that issues a huge amount of unimportant alerts becomes a larger danger to an organisation than no IDS 
at all. ".....   What appears to be unimportant information at one moment in time may turn out to be very important 
later on...   Take for instance the information NASA had about the debris that hit the shuttle during launch.  This 
event was detected by various cameras (sensors) monitoring the liftoff, however it seemed to be of no significant value 
at the time.  This was not because the data was not valuable, but rather because the interpretation was not accurate.   
Most information has value if it is understood and interperted properly.  However, data becomes significantly less 
useful when the proper tools and/or skills are not used to interpret the information. 
 
-Brian
 

        -----Original Message----- 
        From: Fergus Brooks [mailto:fergusb () evolve-online com] 
        Sent: Wed 6/4/2003 11:47 PM 
        To: focus-ids () securityfocus com 
        Cc: 
        Subject: RE: False Positives
        
        


        10 cents:
        
        There isn't an IDS system that will not report "false positives"
        regardless of the definition. While selling IDS systems we use tools
        like the excellent IDS Informer to prove the concept. Being as these
        tools are not actually attacking but testing, and they report an attack,
        then they are generating a *false positive*
        
        I believe this thread was about eliminating them, whatever they are, and
        this is the crucial thing that my customers ask me and also my key
        interest when looking at products to recommend.
        
        Any IDS system that issues a huge amount of unimportant alerts becomes a
        larger danger to an organisation than no IDS at all. This is because the
        business will say they have an IDS however the alerts (good, bad,
        irrelevant) are not monitored because 99 times out of 100 (being
        generous...) they are rubbish.
        
        In our region (Asia) I would go as far as to say that 95% of the IDS
        systems that are implemented are not delivering.
        
        Event correlation is a start - baselining, correlation, and consistent
        tweaking and analysis are, IMHO, the best way to eliminate *false
        positives*
        
        Always looking for better ways though?
        
        
        
        -----Original Message-----
        From: Steven Richards [mailto:srichards () netscreen com]
        Sent: Thursday, 5 June 2003 4:19 AM
        To: focus-ids () securityfocus com
        Cc: 'Harshul Nayak (ealcatraz)'; Andi Hess
        Subject: RE: False Positives
        
        
        
        -----BEGIN PGP SIGNED MESSAGE-----
        
        <insert standard vendor disclaimer here>
        
        I would like to offer up a couple of definitions of terms for
        discussion.
        
        We in the security space, more specifically the IDS/P space should agree
        on some standard language.
        
        
        In my opinion, these terms should be considered:
        
        False Positive=   Sensor is supposed to be looking for 'xyz' and what
        actually goes across the wire is 'abc' which generates an alert.
        
        Non-Security Event=   Sensor is looking for 'XXX' it sees 'XXX' go
        over the wire, *but* it is not an actual "Security Event" because the
        corporate security policy and/or the system configurations are
        intentionally not configured to disallow 'XXX'.  For example: you
        configure your systems to allow Null Login NetBIOS Sessions (for
        whatever reason) and your corporate security policies do not disallow
        this.  The sensor sees this traffic and it generates an alert.  It
        actually happened on your network.  It's just that you don't *care*
        about it.
        
        
        
        
        > -----Original Message-----
        > From: Harshul Nayak (ealcatraz) [mailto:harshul () ealcatraz com]
        > Sent: Wednesday, June 04, 2003 1:11 PM
        > To: Andi Hess
        > Cc: focus-ids () securityfocus com
        > Subject: RE: False Positives
        >
        >
        > Hello Andi,
        >
        > It's quite often many people use the term "false positive"
        > for the tests
        > conducted by you,
        > With due respect to all, would like to share with reference
        > to the article
        > by Marcus J. Ranum for ICSA Labs IDSC :
        >
        > Your mail is referring to :
        >
        > * False Attack Stimulus - A stimulus that causes an IDS to
        > trigger an alarm
        > when no actual exploited attack has
        > occurred. False attack stimuli generate false positives; and
        > are frequently
        > seen during badly designed IDS tests or
        > when attackers attempt to overload an IDS' alert processing
        > capability using
        > a tool such as Stick. Many scanning
        > tools generate false attack stimuli. For example, if a
        > vulnerability assessment tool connects to a web server and
        > issues a "GET" for a known-vulnerable CGI-bin script, it is
        > not the same
        > thing as when a hacking tool connects and
        > exercises the complete attack via the same script.
        > Depending on the application protocols in use it may be
        > difficult for the
        > IDS to distinguish a stimulus that looks for a
        > vulnerability from a stimulus that actually triggers a
        > compromise in the
        > system. False attack stimuli are deliberately
        > used in some IDS testing regimens, attempting to verify the
        > IDS' function
        > without placing real systems at risk. When
        > testing IDS a tester should mix a number of false attack
        > stimuli with true
        > attack stimuli.
        >
        > Here is the definition of "false positive"
        > * False Positive - An alarm generated by an IDS in which the
        > IDS alerts to a
        > condition that is actually benign. In
        > other words, the IDS made a mistake. A typical example of a
        > false positive
        > would be a case when an IDS raises a
        > "SYN flood" alarm because it sees a large number of SYN
        > packets directed at
        > a busy web server and mistakenly
        > concludes it is under attack. Another example of a false
        > positive would be
        > an IDS raising a "SMTP Wiz attack" alarm
        > when it observes the string "DEBUG" in the body of an SMTP message.
        >
        > hope this helps you.
        >
        > -regs
        > Harshul
        > Copyright C 2002 Sintelli
        > http://www.sintelli.com
        > ----------------------------------------------------------------
        > "A good listener is usually thinking about something else."
        >
        > -----Original Message-----
        > From: Andi Hess [mailto:andi_hess () web de]
        > Sent: Tuesday, June 03, 2003 4:13 PM
        > To: focus-ids () securityfocus com
        > Subject: False Positives
        >
        >
        >
        >
        > Hi there,
        >
        > I am new in the field of NIDS and I wonder if the
        > problem of false positives is really this huge as
        > mentioned in several publications.
        >
        > I am considering tools like PCP, Stick (I have never
        > seen them, but read about them) which can be used to
        > generate huge amount of packets and each on triggers an
        > alarm on the victim IDS (a false positive, as the
        > packets are not a real attack).
        > As it has been impossible for me to find any of the
        > above mentioned packet generators - I wonder how the
        > packets look like?
        > Is it possible to differentiate 'artifically' generated
        > false positives from natural ones?
        >
        > Any hint is welcome!
        >
        > Thank you.
        >
        > A.
        >
        >
        >
        >
        > --------------------------------------------------------------
        > --------------
        > ---
        > INTRUSION PREVENTION: READY FOR PRIME TIME?
        >
        > IntruShield now offers unprecedented Intrusion IntelligenceTM
        > capabilities
        > - including intrusion identification, relevancy, direction, impact
        > and analysis
        > - enabling a path to prevention.
        >
        > Download the latest white paper "Intrusion Prevention: Myths,
        > Challenges,
        > and Requirements" at:
        > http://www.securityfocus.com/IntruVert-focus-ids2
        > --------------------------------------------------------------
        > --------------
        > ---
        >
        >
        > --------------------------------------------------------------
        > -----------------
        > INTRUSION PREVENTION: READY FOR PRIME TIME?
        >
        > IntruShield now offers unprecedented Intrusion IntelligenceTM
        > capabilities
        > - including intrusion identification, relevancy, direction,
        > impact and analysis
        > - enabling a path to prevention.
        >
        > Download the latest white paper "Intrusion Prevention: Myths,
        > Challenges, and Requirements" at:
        > http://www.securityfocus.com/IntruVert-focus-ids2
        > --------------------------------------------------------------
        > -----------------
        >
        
        -----BEGIN PGP SIGNATURE-----
        Version: PGP 7.0
        
        iQEVAwUBPt5VYWNGcU7aCIRPAQG9+ggAgnFVJUYL/AL65LrpGGs54SKtbX2uze49
        xOneZcytR5GDi0PROh139SMB6ytrleIQjodtckAxGsZxsH4nASL2XG88FfjyXOkE
        l8gj6QyQoQGCE2FQf8333q1FOkCtT2BSm0YpY1/ekVgeDhZz05iuG++hF5IUPQZg
        l3YMv2Oes1/mW9jOS3MOP4Bggq7xnKb7LBPr7eJFmhW0HZe+AR8knj2aV54OB3+N
        DQhfgrBDKAM+Xqg/cqaG5L5KVejfRue7JEWfTt7kqDBP8hdxejAQJx3G7sb1yj5/
        gN2Kq4SKBP0f1hbBPxmRC2+oEGffy/+hAW4DiMg/NhkZb/IvvPMAqw==
        =wkY4
        -----END PGP SIGNATURE-----
        
        ------------------------------------------------------------------------
        -------
        INTRUSION PREVENTION: READY FOR PRIME TIME?
        
        IntruShield now offers unprecedented Intrusion IntelligenceTM
        capabilities
        - including intrusion identification, relevancy, direction, impact and
        analysis
        - enabling a path to prevention.
        
        Download the latest white paper "Intrusion Prevention: Myths,
        Challenges, and Requirements" at:
        http://www.securityfocus.com/IntruVert-focus-ids2
        ------------------------------------------------------------------------
        -------
        
        --
        This message has been scanned by AVMail.
        
        
        
        -------------------------------------------------------------------------------
        INTRUSION PREVENTION: READY FOR PRIME TIME?
        
        IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities
        - including intrusion identification, relevancy, direction, impact and analysis
        - enabling a path to prevention.
        
        Download the latest white paper "Intrusion Prevention: Myths, Challenges, and Requirements" at:
        http://www.securityfocus.com/IntruVert-focus-ids2
        -------------------------------------------------------------------------------