Re: Announcement: Alert Verification for Snort

["Bill Royds" <broyds () rogers com>]

This is a very interesting point of view because it points to a new way of
creating an IDS.
What we really want is a language that describes the context of a system (in
send of multiple servers, networks, programs etc.) and a language that
describes the possible attacks at level of how those attacks can damage this
system. We can then compile these two into a system vulnerability
description that can optimally respond to data  hitting the system.
   Now the main set of data for creating a particular implementation of an
firewall is to have a systems description of data flows allowed into the
system, a grammar of allowed input. This grammar  can then create a filter
that limits allowed data so that one has closer to an "allow only valid
input, deny the rest" stance for actual data content, not just the present
envelope view of most firewalls.
  The IDS' job is then to analyze that which was denied to determine why and
tune the firewall to better filter in the future.
   The limit to this approach is similar to the concept of anomaly detection
IDS, except that the anomalies are not strictly from context but from a
grammar. The grammar is created by looking at the input/output descriptions
for sostware running on the system (which includes the RFC for protocols
allowed, the scanf/printf  parameter set, "well known" values for buffer
sizes in software etc.).  This grammar view also allows for traffic
normalizers attuned to the system that are protected.
  One integrates the concept of vulnerability scanner, firewall, IDS and
honeypot in a single system.
The vulnerability scanner and signature set create the grammar for the
firewall rule.
The firewall examines all traffic for conformance with the grammar. That
which conforms is passed through to the system, that which fails is passed
through to the honeypot that captures it and increases the contextual
information for the IDS to analyze it (a honeypot running IIS would give the
correct return code for an IIS attack without exposing the corporate data to
the exploit). Essentially the honeypot mirrors the software base of the
system without the vulnerable data set.  True false positives (in Marty's
sense) would point to a need for tuning the firewall to prevent a denial of
service, but contextual false positives would only imply updating the
honeypot.
 This still has the risk of false negatives, but monitoring the reaction of
the real system could add more context information to the grammar to reduce
those as well.



----- Original Message ----- 
From: "Stephen P. Berry" <spb () meshuggeneh net>
To: "Martin Roesch" <roesch () sourcefire com>
Cc: "Christopher Kruegel" <chris () cs ucsb edu>;
<focus-ids () securityfocus com>; <spb () meshuggeneh net>
Sent: Thursday, October 23, 2003 10:33 PM
Subject: Re: Announcement: Alert Verification for Snort


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Martin Roesch writes:

> [...] I  think it's important to make the distinction between "real" false
> positives (i.e. the IDS screwed up) and nontextuals where the IDS has
> done its job, it just needs more information to properly evaluate the
> reality and priority of the event.

Insufficient context is clearly -part- of the problem in many cases,
and I don't really disagree with the above evaluation.

I think the problem[0] is also more fundamental, however, and is based on
a pervading infosec GCE[1] that holds that meaningful conclusions about
large-scale behaviours (i.e., the remote compromise of a network-attached
device) can be made on the basis of individual packets.

This misconception isn't merely the result of misinterpreting
the -scope- of an NIDS event (i.e., what a single signature match
tells you), but rather the -character- of such events (i.e., what
kind of information such events are capable of conveying).

Some time ago while fiddling around with constructing a formal grammar
for event handling for use in some future release shoki, a conceptual
model occurred to me:

NIDS signatures and most other intrusion detection heuristics (i.e.,
regex searches through syslog output) merely serve as a mechanism for
tokenising the raw data stream (packets on the wire or
lines in /var/log/messages, for example).  In other words they are
components of a lexical analysis system, the output of which is a
stream of recognised tokens.

Currently, almost all NIDS systems do -just that-, and rely on the
analyst[2]
to actually parse these tokens.

Expecting such a system to meaningfully predict the actual state of
a network is kinda like grepping through a bunch of C code for
things like `snprintf' and `fork', and expecting to be able to predict
what the program will do.

Evaluating whether a particular event emitted by a NIDS widget is
a `false positive' is a semantic evaluation, not a lexical
categorisation---but all most NIDS produce are lexical analyses.  This isn't
merely a problem of insufficient context---it's a misconception about the
underlying character of the data.

Does this make sense?  I think this is a nontrivial distinction, but
I'm not sure that the above makes it clear.


That all aside, I further think that evaluating the content of a
NIDS event is more complex than testing for the presence or absence of
some quantity(-ies).  I.e., is there an `actual' attack or not, and did
the NIDS catch it or not.

A rigourous discussion of this contention is probably beyond the scope
of this message, but I would suggest that this evaluation is -at least as-
complex as risk analysis, and is probably in fact more complex.  Put
a slightly different way:  you can't really evaluate whether something
is a `false positive' or not except within the context of a threat
model.  Since very few IDSes have any way of representing a threat
model, very few have any way of even -expressing- the notion of
a `false positive'.

Note that this is distinct from the point I made (or tried
to make) above:  if you'll excuse the ornate diction,
the lexical analysis/semantic content distinction is an epistemological
point;  the threat model contention is teleological.  Somewhat more
vaguely, the first distinction I made was largely about the structure
of an IDS and the latter is about conceptual basis(-es) upon which
value judgements are made by that NIDS.


My point?  Asking whether or not a NIDS reporting a signature match for a
given packet is a `false positive' isn't even an insufficiently specified
problem.  It's like asking if OBP[3] is a better OS than SRM[4].  The
question is poorly formed.




- -spb

- -----
0 Or at least the -semantic- problem, here `semantic' meaning
`having to do with semantics' and not the (more common in
mailing list and USENET discussions) `nomenclature quibble'
usage of the term.
1 Gross conceptual error.
2 I.e., a human operator.
3 I (perhaps naively) expect most readers will still recognise
this as Sun's Open Boot Prom.
4 DEC's Systems Reference Manual, one of the boot consoles used
on alphas and, earlier, VAXen.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (OpenBSD)

iD8DBQE/mI9DG3kIaxeRZl8RAgFeAJ9bYT134ePcu/8Vw93QwZVGgBsL6QCeOrpS
vOFUBi9glzLcg7Cj5S0VXTQ=
=3Rw4
-----END PGP SIGNATURE-----

---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023
and use priority code SF4.
---------------------------------------------------------------------------


---------------------------------------------------------------------------
Network with over 10,000 of the brightest minds in information security
at the largest, most highly-anticipated industry event of the year.
Don't miss RSA Conference 2004! Choose from over 200 class sessions and
see demos from more than 250 industry vendors. If your job touches
security, you need to be here. Learn more or register at
http://www.securityfocus.com/sponsor/RSA_focus-ids_031023 
and use priority code SF4.
---------------------------------------------------------------------------