On the definition of false positive - was: Re: location of an IPS

[Evil Adam Smith <eviladamsmith () yahoo com>]

Hey List,

Since Kurt obviously isn’t afraid to correct
others...and I know at least one person on the list
might also benefit from this comment... 

> From Kurt's post below: 
"One the one hand good, that would have been a false
positive technically speaking, otoh that's bad, it
probably should have alerted on that (even if it is a
false positive)."
 
Actually, I believe it would be either a true or false
negative - depending on how you defined the terms. In
this example choose to use true.
For example in the model I'm thinking of:
A false positive is when an attack is detected
(positive), but it wasn't a real attack (false) -
whatever the reason the signature triggered falsely or
some such.
A true positive is when it was detected (positive) and
it was a real attack (true).
A false negative is when it wasn't detected (negative)
and it wasn't a real attack (false) - you could test
for false positives with false negatives (things the
IPS shouldn't ever detect as malicious(valid
traffic)).
Thus, a true negative is a real attack(true) that goes
undetected (negative).

I guess Kurt was thinking intent of the attacker
matters a la an alternative definition of "attack" but
such a definition would be I believe untestable - how
would IDSes, etc. ever be able to establish the intent
of a packet? If I scan myself my ids either detected
it or it did not. 

Semantic quibbles aside I don't see a more useful way
to think about this problem area using only two sets
of two terms and use them in a meaningful practical
way.  

Cheers
eviladamsmith

> "Kurt Seifried" <bt () seifried org>
> 10/19/2005 09:13 PM
> Please respond to
> "Kurt Seifried" <bt () seifried org>
>
>
> To
> "Doug Fox" <dfox168 () hotmail com>,
focus-ids () securityfocus com
> cc
>
> Subject
> Re: location of an IPS
>
>
>
>
>
>
> > I'm sorry for this dumb question, which may have
been answered many
> times.
> > Where should one place an TippingPoint Unity 50
IPS device?  Behind or
> in
> > front of a firewall?
>
> Depends what you want to measure. Broadly speaking
in front of the
> firewall
> means you're measuring attempts, behind the firewall
they are penetrations
> (or do both and then compare them, that way you can
actually tell
> management
> "look we're stoping 90% of detected attacks, now
would you please let me
> tighten the firewall rules so that's 100%?" or
something). One thing to
> remember is to look for outgoing attacks as well,
that's a good indication
> of a compromised host or a hostile user.
>
> > I have a/the TippingPoint behind a Check Point
firewall. Even though we
> > externally and internally port-scanned the
firewall and the IPS many
> > times, the activity log did not contain any record
of the "attacks".
> One the one hand good, that would have been a false
positive technically
> speaking, otoh that's bad, it probably should have
alerted on that (even
> if
> it is a false positive). Sounds like you need to sit
down and do the
> setup/configuration/alerting/whatnot (aka the hard
parts of IDS/IPS).
> Broadly speaking you're saying "it's broken" to
which I can only say
> "bummer. try fixing it."
>
> > What am I missing here?  Any pointers are
appreciated.
> > Thanks,
>
> The dreaded C word comes to mind (consultant), if
your company lacks the
> expertise to set this up buy someones time who does.
>
> -Kurt


                
__________________________________ 
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------