Re: Announcement: Alert Verification for Snort

["Stephen P. Berry" <spb () meshuggeneh net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Martin Roesch writes:

> [...] I  think it's important to make the distinction between "real" false  
> positives (i.e. the IDS screwed up) and nontextuals where the IDS has  
> done its job, it just needs more information to properly evaluate the  
> reality and priority of the event.

Insufficient context is clearly -part- of the problem in many cases,
and I don't really disagree with the above evaluation.

I think the problem[0] is also more fundamental, however, and is based on
a pervading infosec GCE[1] that holds that meaningful conclusions about
large-scale behaviours (i.e., the remote compromise of a network-attached
device) can be made on the basis of individual packets.

This misconception isn't merely the result of misinterpreting
the -scope- of an NIDS event (i.e., what a single signature match
tells you), but rather the -character- of such events (i.e., what
kind of information such events are capable of conveying).

Some time ago while fiddling around with constructing a formal grammar
for event handling for use in some future release shoki, a conceptual
model occurred to me:

NIDS signatures and most other intrusion detection heuristics (i.e.,
regex searches through syslog output) merely serve as a mechanism for
tokenising the raw data stream (packets on the wire or
lines in /var/log/messages, for example).  In other words they are
components of a lexical analysis system, the output of which is a
stream of recognised tokens.

Currently, almost all NIDS systems do -just that-, and rely on the analyst[2]
to actually parse these tokens.

Expecting such a system to meaningfully predict the actual state of
a network is kinda like grepping through a bunch of C code for
things like `snprintf' and `fork', and expecting to be able to predict
what the program will do.

Evaluating whether a particular event emitted by a NIDS widget is
a `false positive' is a semantic evaluation, not a lexical
categorisation---but all most NIDS produce are lexical analyses.  This isn't
merely a problem of insufficient context---it's a misconception about the
underlying character of the data.

Does this make sense?  I think this is a nontrivial distinction, but
I'm not sure that the above makes it clear.


That all aside, I further think that evaluating the content of a
NIDS event is more complex than testing for the presence or absence of
some quantity(-ies).  I.e., is there an `actual' attack or not, and did
the NIDS catch it or not.

A rigourous discussion of this contention is probably beyond the scope
of this message, but I would suggest that this evaluation is -at least as-
complex as risk analysis, and is probably in fact more complex.  Put
a slightly different way:  you can't really evaluate whether something
is a `false positive' or not except within the context of a threat
model.  Since very few IDSes have any way of representing a threat
model, very few have any way of even -expressing- the notion of
a `false positive'.

Note that this is distinct from the point I made (or tried
to make) above:  if you'll excuse the ornate diction,
the lexical analysis/semantic content distinction is an epistemological
point;  the threat model contention is teleological.  Somewhat more
vaguely, the first distinction I made was largely about the structure
of an IDS and the latter is about conceptual basis(-es) upon which
value judgements are made by that NIDS.


My point?  Asking whether or not a NIDS reporting a signature match for a
given packet is a `false positive' isn't even an insufficiently specified
problem.  It's like asking if OBP[3] is a better OS than SRM[4].  The
question is poorly formed.




- -spb

- -----
0       Or at least the -semantic- problem, here `semantic' meaning
        `having to do with semantics' and not the (more common in
        mailing list and USENET discussions) `nomenclature quibble'
        usage of the term.
1       Gross conceptual error.
2       I.e., a human operator.
3       I (perhaps naively) expect most readers will still recognise
        this as Sun's Open Boot Prom.
4       DEC's Systems Reference Manual, one of the boot consoles used
        on alphas and, earlier, VAXen.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (OpenBSD)

iD8DBQE/mI9DG3kIaxeRZl8RAgFeAJ9bYT134ePcu/8Vw93QwZVGgBsL6QCeOrpS
vOFUBi9glzLcg7Cj5S0VXTQ=
=3Rw4
-----END PGP SIGNATURE-----

---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023 
and use priority code SF4.
---------------------------------------------------------------------------