Snort mailing list archives
Re: barnyard & log_unified problem
From: Devin Kowatch <dkowatch () scea com>
Date: Wed, 28 Jun 2006 18:09:21 -0700
Hi, Unfortunatly I can't provide a broken unified log. At least not without sanitizing it, which I think will be difficult in this case. However, I can describe in general terms what my log file looks like, which I think may shed some light on the problem. I tried having barnyard send output to log_pcap from one of the broken files. Obviously I only made it up to the packet with the last good record header. However, the pcap dump was enlightening. In this case the last good UnifiedLog header that I got was for a frag overlap alert from the frag3 preprocessor (probably caused by duplicate packets on my sniffing interface, but that's another story). Note that last time I looked this closely at it, the alert was from the sfportscan preprocessor, so it's not neccessarily a problem with either preprocessor. The packet that was logged was the first fragment, 1500 bytes (caplen is 1514 for the ethernet header), and NFS over UDP. Anyway, when examining the barnyard pcap log with tcpdump -X, I found that part way through there was an HTTP response message (e.g. HTTP/1.1 200 ....). After looking at it, this appears to be a full packet, complete with IP and TCP headers (at least they looked a lot like TCP/IP headers). At this point I went back to the unified log file and started looking at it closer. Below is my attempt at a diagram of what I found in the file, the last good record started at offset 3560 (decimal). 3560: UnifiedLog record header (frag3 alert) 3616: start packet (14 byte ethernet header) 3630: start IP header, followed by UDP header. 4166: start possible IP header 1 (proto TCP, len 606) 4206: start HTTP response 4772: begining of unknown data [1] 4842: start possible IP header 2 (proto TCP, len 606) 5130: Point where barnyard start reading the next unified log record, and then dies because the packet length is too long. 5448: start of more unknown data [2] [1] This is sort of assumed to be the begining of unknown data for two reasons. First it is the byte after the end specified by the first IP header. Second it one byte beyond the string "\r\n\r\n" which is usually a indicator for the end of a HTTP packet. [2] Again this is presumed to be start of the unknown data for the exact same reasons noted in [1]. Now there is one more thing that I should say. In both cases the unknown data (@4772 and @5448) look like they might be a unified log record header. I checked generator ID, signature ID, revision, and priority and all are consistent; the sizes work out, 56 bytes for the record header + 14 bytes for the ethernet header; the time and event IDs seem in line with the good record headers; and packet lengths are correct. I also went back and checked the data leading up to the possible IP header @4166. Starting 70 bytes prior, @4096, I see what again appears to be a valid record header (again for the same reasons as noted above). Were I to venture a guess, I would say that snort appears to be writing only part of the data from one event, and then writing more event data after that short write. However, because the first write was short, barnyard is getting confused and ends up out of sync with the file. Hopefully this helps in tracking the problem down. If you have any questions let me know. Thanks, -devink On Wed, Jun 28, 2006 at 03:18:39PM -0600, Bamm Visscher wrote:
Deviin, I've seen multiple reports of this but never seen it myself. I cc'd the barnyard-users list on my reply. Maybe Andrew can give us some input. Maybe if someone could send a borken unified log to the snort dev team? Bammkkkk On 6/28/06, Devin Kowatch <dkowatch () scea com> wrote:Hi, I've had barnyard dying on me occasionally, while reading snort's log_unified output. Under snort 2.4.3 Barnyard would die with an "Invalide packet length" error. After some investigation, it was looking like barnyard was reading the file correctly (using od to dump the file and matching that to what barnyard was reading). So I figured the problem with either that snort was corrupting the file, or there was an incompatability between barnyard and snort. In any event, I upgraded to snort 2.6.0 to see if that fixed the problem. Now under snort 2.6.0 Barnyard is dying with "FATAL ERROR: Out of memory (wanted 4230306464 bytes)". Using gdb this appears to be happening in the same function that the "Invalid packet length" error message happens in (specifically LogDpReadRecord). In this case the cause appears to be the same as before. Which is to say that the caplen field of the UnifiedLog record header is way to large [1]. I've seen some other reports of this problem, but haven't found any resolution to it. I'm hoping that is just because I haven't looked in the right places, but if not, then hopefully I can be of some help figuring out what is going wrong. I get the same error if I run barnyard in daemon mode using the sguil ouput plugin, or if I run it in one shot mode using the default config file. All of this is running on an Intel P4 using CentOS. My snort output configuration is: output alert_unified: filename snort.alert, limit 512 output log_unified: filename snort.log, limit 512 Any help would be greatly appreciated. Thanks, -devink [1] Barnyard has a sanity check which is supposed to catch excessively large caplens. When that sanity check fails it leads to the "Invalid packet length" error message. In this case the sanity check is not failing because barnyard is converting SnortPktHeader.caplen from an unsigned value to a signed value prior to performing the sanity check. Because the value in this case is so large, when the sanity check is performed, the caplen value is negative, and thus passes the sanity check. After that it tries to allocate a bunch of memory and fails. The signed/unsigned thing is probably a separate bug in barnyard, but I'm not completely sure where to report it. Or is this the correct forum? -- Devin Kowatch Sony Computer Entertainment of America dkowatch () scea com Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users-- sguil - The Analyst Console for NSM http://sguil.sf.net
-- Devin Kowatch Sony Computer Entertainment of America dkowatch () scea com Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
Current thread:
- barnyard & log_unified problem Devin Kowatch (Jun 28)
- Re: barnyard & log_unified problem Bamm Visscher (Jun 28)
- Re: barnyard & log_unified problem Devin Kowatch (Jun 28)
- Re: barnyard & log_unified problem Bamm Visscher (Jun 28)