Re: Binary log capture looks incomplete.

[beenph <beenph () gmail com>]

On Fri, May 24, 2013 at 3:40 PM, Shields, Joseph (NIH/NIEHS) [C]
<joseph.shields () nih gov> wrote:
> I don't think the limit comes into play when the packets count value is set to zero.  If I am understanding the 
> manual correctly:
>
> You can disable this packet limit for a particular rule by adding a packets metric to your tag option and setting its 
> count to 0 (This can be done on a global scale by setting the tagged packet limit option in snort.conf to 0).
>
> I have each rule with this setting which ought to cause any hit to be captured in its entirety:
> tag:session,0,packets,1000,seconds;
>
> I tried to get the "logto" option to work yesterday afternoon.  I removed the -b startup option and have -l 
> /var/log/snort, yet it still looks like the captures are going to the same capture file as before.
>
>  -rw-r--r--   1 snort snort     22259 May 24 11:31 alert
> -rw-------   1 snort snort  17228586 May 22 16:12 snort.log.1369233576
> -rw-------   1 snort snort  34636725 May 24 10:11 snort.log.1369347708
> -rw-------   1 snort snort   7623535 May 24 11:33 snort.log.1369408065
>
>  I'm not sure if the logto files need to exist before hand or if snort would create them.  I went ahead and created 
> the logto files this morning and set the ownership to snort.  They still are not being written to.  Here is the 
> startup command I am using and the logto setting I have within one of the rules with hits:
>
> logto:"/var/log/snort/logto/sid2000419.log";
>
> ./snort -A fast -d -D -i em3 -u snort -g snort -c /etc/snort/snort.conf -l /var/log/snort2 -G 3
Do you want to log in unified2 format or tcpdump format?

If you want unified2 do not use -A fast or -b at the command line level :)


> The problem I am trying to resolve at the moment is the apparent double logging to the snort.log.1369408065 file for 
> two rules I am interested.  What I have noticed is that if they both alert, then I get the session to be written to 
> the snort.log file both at the same time.  I commented out one of the rules this morning that was nearly always 
> getting alert at the same time as another rule.  Looking at the snort.log.nnnnnnnn file I no longer see the double 
> logging I was getting yesterday.
>
> At this point I am wondering what is the best way to setup session captures?  I need complete session captues when a 
> rule gets flagged.   It sure seems to me that if two different rules flag a session, it should only be logged to the 
> binary log file once, not twice or however many times there are rules that set an alert.
>
>
> Brian
>
>
>
>
> -----Original Message-----
> From: beenph [mailto:beenph () gmail com]
> Sent: Thursday, May 23, 2013 8:25 PM
> To: Shields, Joseph (NIH/NIEHS) [C]
> Cc: James Lay; snort-users () lists sourceforge net
> Subject: Re: [Snort-users] Binary log capture looks incomplete.
>
> By default snort tag packet limit is 256 #define MAX_TAG_NODES 256
>
> Unless you use the configuration option tagged_packet_limit to up that value.
>
> -elz
>
>
> On Thu, May 23, 2013 at 6:35 PM, Shields, Joseph (NIH/NIEHS) [C] <joseph.shields () nih gov> wrote:
> > James,
> >    My understanding from reading the Snort manual is that in the rule these settings affect logging:
> > 1) flowbits:set,tagged;     Tells snort that the session should be logged when this rule is positive.
> > 2) tag:session,0,packets,1000,seconds;      Tells snort to not limit the size (set to zero) and to log for up to 
> > 1000 seconds.
> >
> > I am not setting the dump type format, so it ought to use the default which I believe is tcpdump.
> >
> > I am now trying to run snort without the -b option and I have setup
> > individual files for each rule by setting the logto option For
> > example, logto:"/var/log/snort/logto/sid2000419.log";
> >
> > I don't have any hits yet to see if this is working.
> >
> > Brian
> >
> > -----Original Message-----
> > From: James Lay [mailto:jlay () slave-tothe-box net]
> > Sent: Thursday, May 23, 2013 6:21 PM
> > To: snort-users () lists sourceforge net
> > Subject: Re: [Snort-users] Binary log capture looks incomplete.
> >
> > Joseph,
> >
> > How are you logging this...with:
> >
> > output log_tcpdump: tcpdump.log
> >
> > or
> >
> > output unified2: filename unified
> >
> > ?
> >
> > As I understand it, (someone please correct me if I'm way off) Snort will capture the packet (or re-assembled 
> > packets) that fired the rule.
> > Snort won't capture the whole conversation or file.  For example, VRT rule 25513 will capture the packet(s) that 
> > match content:"application/octet-stream"; and content:"MZ";, but won't capture the entire file.  Hope that sheds 
> > some light on it.
> >
> > James
> >
> >
> > On 2013-05-23 14:05, Shields, Joseph (NIH/NIEHS) [C] wrote:
> > > I sent the below question to the user list yesterday but did not see
> > > any replies to it. I think I have an idea about what is going on here.
> > > I had my own rule to look for a specific event. In watching the
> > > snort-sigs user list I saw last week a rule that looked like it might
> > > be more effective, so I added it to the rule set. What I've noticed
> > > when trying to look at the binary captures this week is that it looks
> > > like the transmission is being restarted. I get the same beginning
> > > sequence number for the internal computer showing up in the log file
> > > capture. I think that what is happening is I have both rules tagged
> > > to capture any hits (flowbits:set,tagged;
> > > tag:session,0,packets,1000,seconds;). I think both rules are
> > > triggering a binary capture for the same session and so the binary
> > > log file is being written to essentially twice. I noticed today that
> > > I had one session alert on three different rules. I can see from the
> > > capture that I have the starting packet sequence number show up three
> > > times (I had 3 different rules that alerted). I think this is causing
> > > me to not get the complete file capture at all.
> > >
> > >  The question is what to do about this? My main concern is to not
> > > miss a network traffic of interest. I don't want to disable rules
> > > rules because they may not all fire. (I'm thinking two rules may
> > > ensure the most coverage. I've had cases where 1 rule will fire
> > > without the other being triggered). I think it is possible to have
> > > Snort stop any further rule testing if a rule gets triggered. That
> > > would be fine (setup suggestions are welcomed). I am thinking to I
> > > could turn off the binary log capture and use the "logto" option
> > > within each rule to point it to a separate file (or folder to use)? I
> > > suppose it would be a file much as what the binary capture does where
> > > multiple hits that day would all end up in the same file.
> > >
> > >  Here is how I start the snort monitoring process:
> > >
> > > /usr/sbin/snort -A fast -b -d -D -i em3 -u snort -g snort -c
> > > /etc/snort/snort.conf -l /var/log/snort -G 3
> > >
> > > Brian
> > >
> > > I SENT THIS
> > >  SUBJECT: Binary log capture looks incomplete.
> > >
> > > In examining a Snort session binary capture resulting from a rule
> > > being triggered I noticed that the transfer was attempted 4 different
> > > times. The amount of data being sent each time was longer before a
> > > reset occurred. The HTTP session parameters noted a content length of
> > > 191K. It is clear the transfer was incomplete as the largest file
> > > size I had to work with was about 24K. What I observed in looking at
> > > the session packets is that the sequence numbers are not complete in
> > > ascending order. Here is how I have the rule tagged in the rules file
> > > so as to perform a capture.
> > >
> > > flowbits:set,tagged; tag:session,0,packets,1000,seconds;
> > >
> > > I am wondering if there is a problem in: 1) the logging being done by
> > > Snort; 2) our mirrored tap is dropping packets; or 3) or if perhaps
> > > the transfer was messed up for whatever reason so I do not need to do
> > > anything. In looking at the sequence numbers I'm thinking a reset
> > > should have occurred when the sequence number of the receiving system
> > > went from 484 to 504 (snipett below from full list further down in
> > > this message). Yet, the dump continued after 504 was received (no
> > > reset). This has me tending to believe more data was sent (nothing
> > > wrong with Snort and is why the logging continued) and very likely
> > > our network tap (dropping packets) has a problem. I thought I would
> > > see what others have seen and would recommend.
> >
> >
> > ----------------------------------------------------------------------
> > -------- Try New Relic Now & We'll Send You this Cool Shirt New Relic
> > is the only SaaS-based application performance monitoring service that
> > delivers powerful full stack analytics. Optimize and monitor your
> > browser, app, & servers with just a few lines of code. Try New Relic
> > and get this awesome Nerd Life shirt!
> > http://p.sf.net/sfu/newrelic_d2d_may
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest Snort news!
> >
> > ----------------------------------------------------------------------
> > -------- Try New Relic Now & We'll Send You this Cool Shirt New Relic
> > is the only SaaS-based application performance monitoring service that
> > delivers powerful full stack analytics. Optimize and monitor your
> > browser, app, & servers with just a few lines of code. Try New Relic
> > and get this awesome Nerd Life shirt!
> > http://p.sf.net/sfu/newrelic_d2d_may
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest Snort news!

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!