Re: format of unified log file?

[Rob Baxter <rbaxter () xapiens net>]

Thanks Joe, that was a great help. If anyone else is interested I found 
the Logtopcap utility (http://dragos.com/logtopcap.com) a little easier 
to follow than the snort src when it came to figuring out the alert & 
log file formats, just because it is much less complicated. Using that 
as a guide it was pretty easy to write a parser for both the alert and 
log output files.

What I'd like to be able to do now is to combine the alert and log files 
into one truly unified output mechanism. In other words take all the 
alert header information from the alert file (signature, ip information, 
protocol, flags, etc) and combine it with the packet capture when 
available. Knowing when a packet capture is available is what seems to 
be tricky. It appears that the EventID field of the alert and logs can 
be used to correllate them, but I'm guessing there isn't always a 1:1 
correspondence with alerts and logs. If that assumption is correct, is 
there any way of determining programatically whether a given alert will 
generate a packet capture? I'm thinking that things like preprocessor 
alerts (or anything with a generator id != 1) will likely not have an 
associated packet dump.

If there is no way to definitively determine whether or not a packet 
capture will be present, I am thinking a reasonable solution would be to 
place each alert in a queue as it is read and keep it there until either:

a) a matching log entry is read
b)  a log entry with a higher EventID is found (presumably meaning there 
is no log entry for that alert)
or c) a predetermined time threshold is exceeded (after which we will 
assume no log entry is coming)

Anyway, that is my initial stab at solving the problem after looking at 
the output files. Anyone with a more in depth knowledge of how snort 
handles alerts & logs care to comment?

TIA,
</rob>

Joe Patterson wrote:

> The way that I figured it out was to look at the spo_unified.c, and parse
> out the various struct's within that.  Yes, it's C, but it's not
> particularly complicated to logically parse out the structure.  Look at
> these structures: UnifiedLogFileHeader, UnifiedLog, Event, SnortPktHeader.
> Basically, the log file starts with a UnifiedLogFileHeader, and then has a
> bunch of UnifiedLog's, which consist of an Event struct, 32 bits of flags,
> and a SnortPktHeader struct. (and then, I believe, caplen worth of actual
> payload data), followed by a new UnifiedLog.
>
> -Joe
>
>  
>
> > -----Original Message-----
> > From: snort-users-admin () lists sourceforge net
> > [mailto:snort-users-admin () lists sourceforge net]On Behalf Of Rob Baxter
> > Sent: Friday, February 11, 2005 8:04 AM
> > To: snort-users () lists sourceforge net
> > Subject: [Snort-users] format of unified log file?
> >
> >
> > I apologize if this has been covered before, but I am looking to write a
> > utility to parse the binary unified log files produced by snort.
> > According to the snort Users Guide the binary log file format is
> > described in the spo_unified.h header file, however when I look at that
> > file I don't see anything resembling a file format desc (in fact the
> > file has < 30 including comments). Am I looking in the wrong place
> > (src/output-plugins directory of the 2.3.0 distribution). Would I find
> > the description in an older revision of this file (appears that I'm
> > looking at v1.9), or where else might I find the documentation on how
> > that file is layed out? I might be able to figure it out by looking at
> > the input processors for mudpit and/or barnyard, but my C is terribly
> > rusty so I'd prefer to have some documentation. If anyone can point me
> > in the right direction it'd be greatly appreciated. thanx,
> >
> > </rob>
> >
> > ---------------------------------------------------
> > Robert M. Baxter
> > Sr. Security Analyst
> > Xapiens Corporation
> > ---------------------------------------------------
> >
> >
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> >
> >
> >    
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>  



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users