RE: IDS Signature Confidence

["Mike Murray" <mmurray () ncircle com>]

I wanted to spark some discussion here - understanding the success of a
signature a priori is a difficult task that is faced by all
signature-based vendors.  At nCircle, we're a vulnerability signature
developer, and we have similar problems - specifically, how do you know
that things are going to work as well in the real world as they do in
the lab?

I imagine that some of the tools that we've created for understanding
vulnerability signatures would apply to IDS signatures as well.  A good
example of this is the idea of signature precision - specifically, how
closely the information that the signature is based on is related to the
actual incidence of the event.   (An example - basing buffer overflows
detection on parts of NOOP sleds or shellcode is not that precise, given
the ability to use a polymorphic shellcode engine).

I wrote up signature precision for VM on our blog, and published the
whitepaper there:
http://blog.ncircle.com/archives/2005/05/vulnerability_p.htm

Perhaps creating more tools like this for IDS signatures would lead to
the type of confidence metric that Raffy's looking for...

-M

> -----Original Message-----
> From: THolman () toplayer com [mailto:THolman () toplayer com] 
> Sent: Thursday, July 21, 2005 5:53 AM
> To: raffy () raffy ch; focus-ids () lists securityfocus com
> Subject: RE: IDS Signature Confidence
>
> Hi Raffy,
>
> If a DoS attack is made up of valid traffic, then a NIDS 
> signature isn't going to pick it up.
> You need to establish whether or not incoming traffic from 
> individual IPs meets acceptable transaction rates, and this 
> is really a job for a rate-based IPS.
>
> Regards,
>
> Tim
>
> -----Original Message-----
> From: Raffael Marty [mailto:raffy () raffy ch]
> Sent: 21 June 2005 00:00
> To: focus-ids () lists securityfocus com
> Subject: IDS Signature Confidence
>
> I was thinking about this following problem: Assume you have 
> an NIDS signature looking for DoS attacks. In most of the 
> cases I don't trust the NIDS reporting on a DoS attack. A lot 
> of the DoS sigs just look at some bytes on the wire and tell 
> me that there is a DoS attack going on. However, I need some 
> more evidence that my services are indeed not accessible 
> anymore.  Some signatures on the other hand are very specific 
> and you can trust them with whatever they report.
> Now this brings me to my question:  How do you guys decide 
> how much confidence you put in a certain IDS signature? And I 
> am not talking about prioritizing the event. I am talking 
> about assigning a "success"
> or "possible success" to signatures.
>
>   -raffy
>
>
> --
>   Raffael Marty, GCIA, CISSP                     
> raffael.marty () arcsight com
>   Senior Security Engineer                     Content Team @ 
> ArcSight Inc.
>   5 Results Way             Cupertino, CA 95014              
> (408) 864-2662
>
> --------------------------------------------------------------
> ------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world 
> attacks from CORE IMPACT.
> Go to 
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> --------------------------------------------------------------
> ------------
>
> --------------------------------------------------------------
> ----------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world 
> attacks from CORE IMPACT.
> Go to 
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> --------------------------------------------------------------
> ----------

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------