RE: Views and Correlation in Intrusion Detection

["Kohlenberg, Toby" <toby.kohlenberg () intel com>]

This has turned into quite an interesting thread! I'm yet another of
those
people who saw this as an issue a couple years ago and have been trying
to
figure out a good solution for quite a while. Since I'm not a code
monkey I've
been looking at published tools and commercial products mostly.

The main thing I wanted to comment on is the (mild) debate about what 
correlation is supposed to give you. From a traditional approach, a
correlation
is something that is identified as having a connection to whatever
you're looking
at. 
> From the IDS space, I'd argue that the primary goal of correlation
should be to
either increase the strength (make real events more visible) of a signal
or 
decrease (make false positives/unimportant events less visible) the
strength of your
noise.

This ends up breaking out into a lot of small activities-
1. looking for known patterns of events that confirm something bad
really is 
happening and prioritize those events
(scan followed by attack followed by communication that shows
compromise)
2. looking for unusual patterns that suggests something subtle is going
on
(one port hit on a box every tuesday morning at 2am from a different IP
but the same
source port and ISN and TTL)
3. keeping track of events that are just noise by themselves but might
be useful when
correlated with other things
(random port probes against your firewall, zone transfers, most of
Snort's ICMP alerts,
BlackICE's "port probe" alerts, etc...)
4. looking for known patterns that confirm a false positive is occuring
and suppress
those events
(When you see 50 "ICMP Echo replies without request" alerts from
BlackICE and they 
are coming from your Exchange server, you know it is probably a false
positive).
.
.
.

There are many more but those are some examples. The basic summary is
that you want to
provide a force multiplier for the human analysts and give them the
information they need
when they need it and present it in such a fashion that they can be as
effective as 
possible.
If you look at correlation tools with an eye towards providing this,
you'll end up with
a much more useful tool.

All opinions are my own and in no way reflect the views of my employer.

thanks,
toby

> -----Original Message-----
> From: David Markle [mailto:davidmarkle () comcast net] 
> Sent: Tuesday, June 17, 2003 7:05 PM
> To: Stephen P. Berry
> Cc: Blake Matheny; focus-ids () securityfocus com
> Subject: RE: Views and Correlation in Intrusion Detection
>
>
> From a purest standpoint, I'll agree with "ONLY" being a little strong
> (below) .... ;)
>
> However, the greater question is how do you 1.) capture and 
> 2.) disseminate
> the "sixth" packet from real traffic ?   Your signature based 
> IDS obviously
> did not know or alert on the "sixth" packet.  Based on the 
> specifics of the
> sixth packet (which are completely uncertain at this point), 
> it may be one
> of several possibilities (which I will not get into for fear 
> of ultimate
> digression).
>
> The point here is .... the correlation mechanism does not replace the
> analyst.  It helps reduce the "noise" so the analyst can do a 
> better job at
> analyzing other anomalous activities (like the "sixth" 
> packet, if at all
> even possible).  Having certain data available at, or before 
> the time of
> incident response proves more efficient than waiting to 
> contact the SA in
> India at 2:30 AM local time.
>
> Finally, you referenced IDMEF in your [0] footnote that was a 
> bit confusing.
> Please elaborate on your comment.
>
>
> -----Original Message-----
> From: Stephen P. Berry [mailto:spb () meshuggeneh net]
> Sent: Tuesday, June 17, 2003 8:31 PM
> To: DAVID MARKLE
> Cc: Blake Matheny; focus-ids () securityfocus com; spb () meshuggeneh net
> Subject: Re: Views and Correlation in Intrusion Detection
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> DAVID MARKLE writes:
>
> > An IDS alert is ONLY relevant if the
> > firewall permits the traffic through.  To further the comment, and
> > attack signature tripped for (known attack) xyz, is ONLY 
> relevant when
> > the attacked host is vulnerable to xyz.  This is the ultimate job of
> > correlation.  If the above surrounding conditions are true, the
> > severity of the attack becomes increased to critical, otherwise it is
> > informational only.
>
> I disagree.  If you see six packets from a single source and 
> five of them
> match five discrete attack signatures and your NIDS doesn't tell you
> anything about the sixth, the smart money says that someone just tried
> five attacks you know about and one you don't.  If you're ignoring the
> five (because you know you're safe from them), you just 
> missed the sixth
> (which is the one you're going to get paged about a couple 
> hours later)[0].
>
> If you're about to suggest that disambiguating this sort of situation
> isn't something that most NIDS products do well (or, indeed, at all),
> ya got me there.  But, alas, we have not yet found a way to convince
> the blackhats to only attack us in such ways as we find convienient
> to monitor[1].  The question you've got to ask yourself is what your
> NIDS is there for:  to behave such that it does not 
> inconvienience your
> incident analysts, or to behave in such a way as to catch the maximum
> number of bad guys[2].
>
> - -spb
>
> - -----
> 0     Random aside:  try to express what I've just described using the
>       IDMEF.
> 1     Certainly not in the general case.  Honeynets and such things
>       undoubtedly work for some situations.
> 2     Of course this is a min/max problem, and neither extreme is
>       optimal.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.1 (OpenBSD)
>
> iD8DBQE+77LKG3kIaxeRZl8RAsZkAKCNkPAuIk8PwHWWyyTFGL97g/28VQCghDsJ
> ufkLX5efYFmRWacwHCtUKQ8=
> =cRIB
> -----END PGP SIGNATURE-----
>
>
> --------------------------------------------------------------
> -----------------
> Attend the Black Hat Briefings & Training, July 28 - 31 in 
> Las Vegas, the 
> world's premier technical IT security event! 10 tracks, 15 
> training sessions, 
> 1,800 delegates from 30 nations including all of the top 
> experts, from CSO's to 
> "underground" security specialists.  See for yourself what 
> the buzz is about!  
> Early-bird registration ends July 3.  This event will sell 
> out. www.blackhat.com
> --------------------------------------------------------------
> -----------------

-------------------------------------------------------------------------------
Attend the Black Hat Briefings & Training, July 28 - 31 in Las Vegas, the
world's premier technical IT security event! 10 tracks, 15 training sessions,
1,800 delegates from 30 nations including all of the top experts, from CSO's to
"underground" security specialists.  See for yourself what the buzz is about!
Early-bird registration ends July 3.  This event will sell out. www.blackhat.com
-------------------------------------------------------------------------------