Re: Intrusion Detection Evaluation Datasets

[Paul Palmer <b.paul.palmer () gmail com>]

On Fri, Mar 13, 2009 at 11:13 AM, Zow Terry Brugger <zow () acm org> wrote:

> The core point I was trying to make is that I haven't seen any attacks
> of interest that one signature based IDS could detect that another
> couldn't. I say attacks of interest because I am aware of some DoS
> attack detection available on some systems, which is only really
> useful on an IPS, because I don't need my IDS to tell me that some
> punk is using a botnet to hammer my systems (also, those DoS attack
> detectors -- at least the good ones -- are not signature based). Now
> if anyone would like to educate me about some signature-based
> technologies that can detect attacks my Snort system can not, I'd be
> eager to learn (as I'm sure many others would, as well). If you want
> to hock your own product, please feel free to contact me off list.

In one aspect, it comes down to how much advance information the IDS
requires the signature writer to have to make a correct assessment.
So, most every IDS should be able to recognize an attack when the
signature writer has perfect knowledge of the attack in advance. That
is to say, if the signature writer knows the exact sequence of bytes
an attacker will attempt to use to attack a system, he should be able
to write a signature to detect that exact attack with no false
positives on most any IDS. Of course, with that level of specificity,
even the smallest changes in the attack render the signature
ineffective.

The better IDS products allow signature writers to write signatures at
higher levels of abstraction in which less specific knowledge of the
attack is required in advance. So, for example, in some (well, at
least one) IDS products, the signature writer can write a single
signature to recognize attempts to exploit a vulnerability in a data
structure embedded within a Quicktime Movie file even before he knows
how the attacker will encode the exploit. This signature writer does
not care what the shell code will look like. He does not care where in
the Movie file the data structure will be. He does not care that the
Movie will be downloaded over HTTP, FTP, or Messenger, or as an e-mail
attachment. He does not care if it will be compressed and chunked over
HTTP. The IDS abstracts all of that away. And yet, the signature is
very likely to be effective even though the exploit writer hasn't
crafted his exploit yet. At the other extreme, in some other IDS
products, the signature writer often has to wait until the exploit is
released to locate unique patterns peculiar to that exploit as the IDS
does not provide sufficient levels of abstraction. In extreme cases, I
have seen signature writers for some products write dozens (in one
memorable case, even hundreds) of signatures in an attempt to provide
adequate coverage for a vulnerability only to see them ineffective a
short time later when the exploit appeared.

Paul