Re: Unified2 file corrupt?

[Zach Hatsis <Zach.Hatsis () maverik com>]

It ended up being -L flag when I ran snort that caused the file to be written in pcap format. Thanks for getting back 
to me tho! Cheers.


From: Bhagya Bantwal [mailto:bbantwal () sourcefire com]
Sent: Wednesday, November 13, 2013 8:26 AM
To: Zach Hatsis
Cc: snort-devel () lists sourceforge net
Subject: Re: [Snort-devel] Unified2 file corrupt?

Hello Zach,

Have you tried with tools/u2spewfoo?

Thanks!
-B

On Mon, Nov 11, 2013 at 2:19 PM, Zach Hatsis <Zach.Hatsis () maverik com<mailto:Zach.Hatsis () maverik com>> wrote:
Hello,

I believe I'm running into issues with snort generating a corrupt unified2 output to my snort logs. I am running 
Snort-2.9.5.5 on CentOS6.4 64 bit .  I compiled it following this guide: 
https://s3.amazonaws.com/snort-org/www/assets/202/snort2953_centos6x.pdf

At first I thought my issue was with Snorby not processing the logs, because I saw data being written to them... then I 
thought it was a barnyard issue, because barnyard wouldn't write any events to the database at all, so the tables were 
all empty.. then I tried running barnyard in batch mode on a log file and got this output:

[root@boulder schemas]# barnyard2 -c /etc/snort/barnyard.conf -o /var/log/snort/snort.u2.1383955664
Running in Batch mode

        --== Initializing Barnyard2 ==--
Initializing Input Plugins!
Initializing Output Plugins!
Parsing config file "/etc/snort/barnyard.conf"
Barnyard2 spooler: Event cache size set to [2048]
Log directory = /var/log/snort/
INFO database: Defaulting Reconnect/Transaction Error limit to 10
INFO database: Defaulting Reconnect sleep time to 5 second
Node unique name is: localhost:eth0

[SignatureReferencePullDataStore()]: No Reference found in database ...
database: compiled support for (mysql)
database: configured to use mysql
database: schema version = 107
database:           host = localhost
database:           user = snort
database:  database name = snort
database:    sensor name = localhost:eth0
database:      sensor id = 1
database:     sensor cid = 8
database:  data encoding = hex
database:   detail level = full
database:     ignore_bpf = no
database: using the "alert" facility

        --== Initialization Complete ==--

  ______   -*> Barnyard2 <*-
/ ,,_  \  Version 2.1.11 (Build 317)
|o"  )~|  By Ian Firns (SecurixLive): http://www.securixlive.com/
+ '''' +  (C) Copyright 2008-2012 Ian Firns <firnsy () securixlive com<mailto:firnsy () securixlive com>>

WARNING: Ignoring corrupt/truncated waldofile '/var/log/barnyard2/barnyard2.waldo'
Processing 1 files...
Opened spool file '/var/log/snort/snort.u2.1383955664'
ERROR: Input file '/var/log/snort/snort.u2.1383955664' is corrupted! (33)
Closing spool file '/var/log/snort/snort.u2.1383955664'. Read 0 records
===============================================================================
Record Totals:
   Records:            0
    Events:            0 (0.000%)
   Packets:            0 (0.000%)
   Unknown:            0 (0.000%)
===============================================================================
Packet breakdown by protocol (includes rebuilt packets):
      ETH: 0          (0.000%)
  ETHdisc: 0          (0.000%)
     VLAN: 0          (0.000%)
     IPV6: 0          (0.000%)
  IP6 EXT: 0          (0.000%)
  IP6opts: 0          (0.000%)
  IP6disc: 0          (0.000%)
      IP4: 0          (0.000%)
  IP4disc: 0          (0.000%)
    TCP 6: 0          (0.000%)
    UDP 6: 0          (0.000%)
    ICMP6: 0          (0.000%)
  ICMP-IP: 0          (0.000%)
      TCP: 0          (0.000%)
      UDP: 0          (0.000%)
     ICMP: 0          (0.000%)
  TCPdisc: 0          (0.000%)
  UDPdisc: 0          (0.000%)
  ICMPdis: 0          (0.000%)
     FRAG: 0          (0.000%)
   FRAG 6: 0          (0.000%)
      ARP: 0          (0.000%)
    EAPOL: 0          (0.000%)
  ETHLOOP: 0          (0.000%)
      IPX: 0          (0.000%)
    OTHER: 0          (0.000%)
  DISCARD: 0          (0.000%)
InvChkSum: 0          (0.000%)
   S5 G 1: 0          (0.000%)
   S5 G 2: 0          (0.000%)
    Total: 0
===============================================================================




So I went a step further back and tried to convert the file using the u2boat tool and got this output:

[root@boulder barnyard2]# /usr/local/bin/u2boat /var/log/snort/snort.u2.1383955664 snortu2-afteru2boat
Defaulting to pcap output.
Error: incomplete record. 2561535 of 33555456 bytes read.


Has anyone else run into this bug?  Thanks!



When I run snort, I run it with these args:
[root@boulder etc]# snort -d -i eth0 -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort -L snort.u2


Below is the unified2 config for /etc/snort/snort.conf:

output unified2: filename snort.u2, limit 128






Zach H


------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net<mailto:Snort-devel () lists sourceforge net>
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!

------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!