Re: Views and Correlation in Intrusion Detection

[DAVID MARKLE <davidmarkle () comcast net>]

Blake, I agree with your sentiments regarding correlation and have more 
to add.

The point of correlation is the value it adds to mostly autonomous, 
unreviewed, and meaningless data. (The folks that disagree with this 
line must have economically independent budgets with staffing 
consisting of superstar (I applaud you)).  Who reviews the firewall 
logs?  I don't.  We have over 500 global firewalls.  The point here is 
(as you stated) AUTOMATION.  But it does not stop there.  That data has 
to be normalized and applied towards something.  The correlation piece 
adds that middleware "something".  An IDS alert is ONLY relevant if the 
firewall permits the traffic through.  To further the comment, and 
attack signature tripped for (known attack) xyz, is ONLY relevant when 
the attacked host is vulnerable to xyz.  This is the ultimate job of 
correlation.  If the above surrounding conditions are true, the 
severity of the attack becomes increased to critical, otherwise it is 
informational only.  There are also netops statistics that should be 
considered security related (and monitored).  Baseline your bandwidth, 
averaged over 12 months.  Normal increases in business offerings are 
roughly 5 percent per month.  Since there was no change control this 
past weekend (to relate), why did you see a spike in bandwidth by 17 
percent ????  Why is tcp 2148 increasing on your global perimeter over 
the past 3 days? These are relevant questions.  Without the collection 
and aggregation of the appropriate data, we run the operations in the 
dark.

With regards to the state of correlation, I still think its an infancy 
issue.  Historically, I believe that the industry (tech folks) has been 
extremely focused on growth development and deployment of the 
technology (firewalls, IDS-(H/N), etc.).  Firewalls have been around 
for awhile and have matured to a point of plateau (mostly).  IDS is now 
in "the growth phase" (with heuristic, anomaly, signature, blah, blah, 
blah), and all that hype.  I really think that the industry had 
recently realizes that we are now overwhelmed with too much data. Now 
everyone is scrambling to catch up .....

David Markle
davidmarkle () comcast net
davidmarkle () elephantfoot org




----- Original Message -----
From: Blake Matheny <bmatheny () mkfifo net>
Date: Tuesday, June 17, 2003 1:32 pm
Subject: Views and Correlation in Intrusion Detection

> Two areas that I have recently been doing research in, are views 
> and their
> connection to correlation techniques. In terms of systems, given 
> some event,
> the information we get about the occurrence of such an event comes 
> to us in
> the form of either a primary or a secondary view. Information 
> about secondary
> views typically come to us from applications such as firewalls and 
> ID systems.
> Primary information usually is received from the application actually
> processing this data for use. For instance, an ID sensor may 
> produce an alert
> about some traffic. However, this is a secondary view of the event 
> and needs
> to be correlated with other, relevant information. So of course 
> firewall logs
> might be checked, to see if traffic actually passed that 
> corresponds to the
> event in question. This is also a secondary view, so a third place 
> is checked,
> the applications logs.
> There are really several issues here. First of all, a tremendous 
> amount of
> time is being spent, trying to correlate all the relevant 
> information. This is
> something that _can_ be automated. Second, the applications logs 
> may not be
> trustworthy. Third, and to me, most importantly, is the fact that 
> this is such
> a 'basic' thing that people using ID systems have to do, and there 
> is no piece
> of software yet that does this.
> So something we have been working on, is a system to deal with 
> this basic
> type of scenario. This will entail data transformations into an 
> intermediarylanguage, an event description language, offline state 
> analysis and several
> other components (there is more information at 
> http://www.nongnu.org/babe/).If you spend some time thinking about 
> everything involved to do this in a
> scalable fashion, it's an enormous task (I said basic, not 
> simple). What I am
> finding frustrating, is that much of the base research has not yet 
> even been
> done. Much of the research that has been done, is either too 
> primitive or too
> impractical to be implemented. Is this due to the infancy and 
> immaturity of
> the field, do people not see this as being feasible and therefor 
> aren'tspending the research time, or is this simply too far down 
> the line? In any
> case, feedback welcome. Thanks.
>
> Cheers,
>
> -Blake
>
> -- 
> Blake Matheny           "... one of the main causes of the fall of the
> bmatheny () mkfifo net      Roman Empire was that, lacking zero, they had
> http://www.mkfifo.net    no way to indicate successful termination of
> http://ovmj.org/GNUnet/  their C programs." --Robert Firth
>
> -------------------------------------------------------------------
> ------------
> Attend the Black Hat Briefings & Training, July 28 - 31 in Las 
> Vegas, the 
> world's premier technical IT security event! 10 tracks, 15 
> training sessions, 
> 1,800 delegates from 30 nations including all of the top experts, 
> from CSO's to 
> "underground" security specialists.  See for yourself what the 
> buzz is about!  
> Early-bird registration ends July 3.  This event will sell out. 
> www.blackhat.com---------------------------------------------------
> ----------------------------


-------------------------------------------------------------------------------
Attend the Black Hat Briefings & Training, July 28 - 31 in Las Vegas, the 
world's premier technical IT security event! 10 tracks, 15 training sessions, 
1,800 delegates from 30 nations including all of the top experts, from CSO's to 
"underground" security specialists.  See for yourself what the buzz is about!  
Early-bird registration ends July 3.  This event will sell out. www.blackhat.com
-------------------------------------------------------------------------------