Re: Views and Correlation in Intrusion Detection

[Randy Taylor <gnu () charm net>]

At 02:25 PM 6/23/2003 -0400, adam.w.hogan wrote:
> It seems to me that this thread and the 'IDS is dead, etc' thread are both 
> coming to same conclusions.  Namely, much more work/research needs to be 
> done in event correlation to efficiently, and effectively, use an 
> IDS.  Now, I think we all have realized this at some point.  I've been 
> wrestling with the idea for some time now, but the scope of this task is 
> enormous, and I'm not sure where to begin.  So I throw this out to the 
> group: how do we start to tackle this problem?

The event correlation problem has been wrestled with since at least 2000 
and discussed in private circles
since at least 1997. I pitched Ron Gula on we later called the "Event 
Correlation Tool" (ECT) in 2000.
Ron liked the idea and set Dan Roelker (now with Sourcefire) and I loose on 
the concept. Ron made
several valuable contributions along the way, and we had a prototype in 
2001. The first I heard
of Riptech's "Calterian" system, which at the time also analyzed Dragon IDS 
data, was 2001. Since then,
folks like ArcSight have come out with products designed to help analysts 
work with the data coming from
IDS and other sources and make better sense of it all. I don't know if any 
of it can be called
"event correlation" - everybody has their own buzzword-compliant acronym 
for "it".

Among the lessons I took from ECT was that it was more an Event 
Significance Tool. In other words,
one can take the individual alerts from any IDS or groups of IDS and derive 
from them a single,
overarching event that describes what the aggregated events mean with a 
high degree of accuracy.
From that, the analyst can then decide what is important and what is not. 
The best reduction
rate we got out of ECT was a ratio of a little over 15,000:1 in our small 
testbed net, but that did not
apply to all cases. Some were of the order of 5:1. Typical was about 30:1 
if I recall. I came to the
conclusion, after considerable discussion with the gang on the Dragon Team 
at the time, that this
type of data analysis was nice, but it wasn't a magic bullet to solve all 
the data fusion problems
analysts face. As far as I know, ECT never made it into a Dragon release. I 
learned a lot anyway
and it was a lot of fun frustrating Dan. ;)

Tom Ptacek, in a post back in May and in a different context, alluded to 
some of the other
elements one needs to rapidly identify signal from noise and take 
appropriate action. Events analyzed
cross-host and cross-protocol, as well as rate-over-time also have to be 
examined.

>   Better statistical analysis?  An open standard to define event data so 
> it can be manipulated easily?  Where does more research need to be 
> done?  Everybody's thoughts would be appreciated.

Everybody I've talked to has a slightly different view of what event 
correlation/data fusion, et al
is with respect to IDS and IPS. I am still learning and I expect I will be 
for a while yet. It is
safe to say, I think, that IPS is dependent on getting the right answer 
from a given dataset. If
an IPS gets it wrong, it's worse than useless. One thing I do know however, 
is that the answer
will be like pornography - I won't be able to describe it exactly, but I'll 
know it when I see it.


> p.s. - I ask not only as a security engineer tasked with making sense of 
> various security tool interfaces and logs, but also as a student looking 
> to do a thesis related to intrusion detection systems (so any other ideas 
> into needed research would be helpful).
>
> -Adam W. Hogan

Hope this helps...or something. 8)

Randy

-----
"We demand rigidly defined areas of doubt and uncertainty!"
--Vroomfondel (in The Hitchhiker's Guide to the Galaxy by Douglas Adams)



-------------------------------------------------------------------------------
Attend the Black Hat Briefings & Training, July 28 - 31 in Las Vegas, the 
world's premier technical IT security event! 10 tracks, 15 training sessions, 
1,800 delegates from 30 nations including all of the top experts, from CSO's to 
"underground" security specialists.  See for yourself what the buzz is about!  
Early-bird registration ends July 3.  This event will sell out. www.blackhat.com
-------------------------------------------------------------------------------