(no subject)

["Paul Melson" <pmelson () gmail com>]

-----Original Message-----
Subject: Re: [Dailydave] News, dumbug, prediction rebuttals.

> If you look at a typical firewall, one sometimes needs to have a nice set
of, say, 400 
> pretty esoteric and ugly regular expressions [which are not fun to write,
not by a long 
> shot] to intelligently parse all the logs into tokens. "Given proper
documentation" is 
> another fantasy; in many cases, there isn't any :-) And don't even get me
started on the 
> convenient interface...

For most correlation tasks, you can only use the pieces of one log format
that are present in other log formats.  Take the example of a firewall log
entry and an IDS log entry.  You extract 6 pieces of information from the
IDS event (timestamp, source addr, source port, dest addr, dest port,
protocol) and 7 pieces of data from the firewall log (the same 6 as the IDS
event plus whether the firewall logged a permit or a deny for that event)
and perform a 1-to-1 match.  The 6 values are numeric, easy to search for,
sort, and match.  Even time offset sloughing is easy to do at this level
(hint to  SIM vendors: pay attention, most of you don't do this but should).
That's one bucketing expression for the IDS logs, one bucketing expression
for the firewall logs, and one more expression to parse the firewall log for
allow/deny.  This should be a very small amount of code.


> OSSIM seems stagnant; I haven't seem any new features for quite some time.
And, just as 
> mentioned by Dave Aitel, device support is a big issue for adoption. If
you read the PIX 6.1 
> logs just fine, there is nothing that tell you that you will deal with PIX
6.2 logs just as 
> fine... Thus, there is a good reason that many SIM softwares cost a bit
more than the above 
> number :-)

I agree that there's a lot of work to be done in terms of device support
before a SIM can pick up market share, but this is one specific thing that
an open source project should be able to make huge advances at.  What's
required is modular parsers, perhaps based on Perl regex or awk (or
something standard and easy to learn).  Then people can write their own and
contribute them back to the project.  Some commercial products offer this
already.

Where the large SIM vendors will continue to succeed in justifying big price
tags, and where open source SIM projects are likely to fail, are in the
large lexicons of expressions necessary to perform event categorization and
threat analysis.  It's easy to extract the information necessary to match
logs from different sources.  To take that information and determine its
relevance is a very different, highly subjective undertaking.

PaulM