Re: Firewall log analysis tools

[Tina Bird <tbird () precision-guesswork com>]

bill royds wrote:

> What I would really like is a repository of Perl regexes for various log
> formats (firewall, router, web server syslog etc.) .
> I have a fair library of Perl routines to create reports, but figuring
> out the proper regexes to read the logs and generate a hash of values to
> analyse is a real pain.

welcome to the wonderful world of log analysis.  the counterpane log
parsing system is regex based, and so a large fraction of the engineering
effort consists (consisted?  i've not been there for a year) of
prioritizing log messages, writing regexes, and testing them in a variety
of ways.  blick.

the closest i've come to building a publicly available library of such
things has been to grab copies of the firewall and IDS "parsing clients"
created as part of the dshield and ARIS (before it went commercial)
collaboration efforts.  i've assumed -- although i haven't had time to
take a look -- that it would be possible to strip out the "parsing" bits
of those things and leverage them to build one big whompin' thing.

there are also a few config files for swatch and logsurfer -- linked to
from the generic parsing tools bit of the loganalysis.org library -- that
are essentially sets of regular expressions.  and of course the config
files in logsentry...

more comments below.

On Wed, 8 Oct 2003, Paul Robertson wrote:

> On Wed, 8 Oct 2003, Vladimir Parkhaev wrote:
>
> > May be we can ask Tina for some space under RegExes & Log parsing category of
> > her webspace. What do you think, Tina?
>
> If Tina isn't interested (hah!,) I'm sure I could set up some space on
> Honor.
"hah" being the operative term.  i've got oh 120 GB of space on that web
server just waiting for libraries of data and regular expressions...

> FWIW, Tina isn't at Counterpane anymore, so your CC probably didn't work,
> but I'm sure she'll see your post to the list.

i'm a comp security officer at stanford now, and still forging ahead on
the log analysis web site as well as the logging infrastructure here.  oh,
and doing time in the microsoft summer internship program, with its
emphasis on interprocess communications and patch management *ugh*

tbird

_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards