Re: barnyard & log_unified problem

[Devin Kowatch <dkowatch () scea com>]

Hi,

Unfortunatly I can't provide a broken unified log.  At least not without
sanitizing it, which I think will be difficult in this case.  However, I
can describe in general terms what my log file looks like, which I think
may shed some light on the problem.

I tried having barnyard send output to log_pcap from one of the broken
files.  Obviously I only made it up to the packet with the last good
record header.  However, the pcap dump was enlightening.  

In this case the last good UnifiedLog header that I got was for a frag
overlap alert from the frag3 preprocessor (probably caused by duplicate
packets on my sniffing interface, but that's another story).  Note that
last time I looked this closely at it, the alert was from the sfportscan
preprocessor, so it's not neccessarily a problem with either
preprocessor.  The packet that was logged was the first fragment, 
1500 bytes (caplen is 1514 for the ethernet header), and NFS over UDP.

Anyway, when examining the barnyard pcap log with tcpdump -X, I found
that part way through there was an HTTP response message (e.g. HTTP/1.1 200
....).  After looking at it, this appears to be a full packet, complete
with IP and TCP headers (at least they looked a lot like TCP/IP headers).
At this point I went back to the unified log file and started looking at
it closer.  Below is my attempt at a diagram of what I found in the
file, the last good record started at offset 3560 (decimal).

3560: UnifiedLog record header (frag3 alert)
3616: start packet (14 byte ethernet header) 
3630: start IP header, followed by UDP header.
4166: start possible IP header 1 (proto TCP, len 606)
4206: start HTTP response
4772: begining of unknown data [1]
4842: start possible IP header 2 (proto TCP, len 606)
5130: Point where barnyard start reading the next unified log record,
      and then dies because the packet length is too long.
5448: start of more unknown data [2]


[1] This is sort of assumed to be the begining of unknown data for two
reasons.  First it is the byte after the end specified by the first IP
header.  Second it one byte beyond the string "\r\n\r\n" which is
usually a indicator for the end of a HTTP packet.

[2] Again this is presumed to be start of the unknown data for the exact
same reasons noted in [1].

Now there is one more thing that I should say.  In both cases the
unknown data (@4772 and @5448) look like they might be a unified log
record header.  I checked generator ID, signature ID, revision, and
priority and all are consistent; the sizes work out, 56 bytes
for the record header + 14 bytes for the ethernet header; the time and
event IDs seem in line with the good record headers; and packet lengths
are correct.

I also went back and checked the data leading up to the possible IP
header @4166.  Starting 70 bytes prior, @4096, I see what again appears
to be a valid record header (again for the same reasons as noted above).

Were I to venture a guess, I would say that snort appears to be writing
only part of the data from one event, and then writing more event data
after that short write.  However, because the first write was short,
barnyard is getting confused and ends up out of sync with the file.

Hopefully this helps in tracking the problem down.  If you have any
questions let me know.

Thanks,
-devink

On Wed, Jun 28, 2006 at 03:18:39PM -0600, Bamm Visscher wrote:
> Deviin,
>
> I've seen multiple reports of this but never seen it myself. I cc'd
> the barnyard-users list on my reply. Maybe Andrew can give us some
> input.
>
> Maybe if someone could send a borken unified log to the snort dev team?
>
> Bammkkkk
>
> On 6/28/06, Devin Kowatch <dkowatch () scea com> wrote:
> > Hi,
> >
> > I've had barnyard dying on me occasionally, while reading snort's
> > log_unified output.
> >
> > Under snort 2.4.3 Barnyard would die with an "Invalide packet length"
> > error.  After some investigation, it was looking like barnyard was
> > reading the file correctly (using od to dump the file and matching that
> > to what barnyard was reading).  So I figured the problem with either
> > that snort was corrupting the file, or there was an incompatability
> > between barnyard and snort.  In any event, I upgraded to snort 2.6.0 to
> > see if that fixed the problem.
> >
> > Now under snort 2.6.0 Barnyard is dying with "FATAL ERROR: Out of memory
> > (wanted 4230306464 bytes)".  Using gdb this appears to be happening in
> > the same function that the "Invalid packet length" error message happens
> > in (specifically LogDpReadRecord).  In this case the cause appears to be
> > the same as before.  Which is to say that the caplen field of the
> > UnifiedLog record header is way to large [1].
> >
> > I've seen some other reports of this problem, but haven't found any
> > resolution to it.  I'm hoping that is just because I haven't looked in
> > the right places, but if not, then hopefully I can be of some help
> > figuring out what is going wrong.
> >
> > I get the same error if I run barnyard in daemon mode using the sguil
> > ouput plugin, or if I run it in one shot mode using the default config
> > file.  All of this is running on an Intel P4 using CentOS. My snort
> > output configuration is:
> >
> >    output alert_unified: filename snort.alert, limit 512
> >    output log_unified: filename snort.log, limit 512
> >
> > Any help would be greatly appreciated.
> > Thanks,
> > -devink
> >
> >
> >
> > [1] Barnyard has a sanity check which is supposed to catch excessively
> > large caplens.  When that sanity check fails it leads to the "Invalid
> > packet length" error message.  In this case the sanity check is not
> > failing because barnyard is converting SnortPktHeader.caplen from an
> > unsigned value to a signed value prior to performing the sanity check.
> > Because the value in this case is so large, when the sanity check is
> > performed, the caplen value is negative, and thus passes the sanity
> > check.  After that it tries to allocate a bunch of memory and fails.
> > The signed/unsigned thing is probably a separate bug in barnyard, but
> > I'm not completely sure where to report it.  Or is this the correct
> > forum?
> >
> > --
> > Devin Kowatch
> > Sony Computer Entertainment of America
> > dkowatch () scea com
> >
> > Using Tomcat but need to do more? Need to support web services, security?
> > Get stuff done quickly with pre-integrated technology to make your job 
> > easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
>
> -- 
> sguil - The Analyst Console for NSM
> http://sguil.sf.net

-- 
Devin Kowatch
Sony Computer Entertainment of America
dkowatch () scea com

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users