Re: Snort only partially alerting

[Frank Calone <fc10011001 () gmail com>]

Fellow Snorters,
    After much testing and with special thanks to Joel, I have an update on
the problem I had with Snort only partially alerting.  I was unaware during
the first few weeks of running snort that the packet datagram was being
truncated for some reason, but not in all cases.  We had an older Snort app
that was flagging alerts which our new server was not.  I was running snort
in the background with this command:

/usr/sbin/snort -A fast -d -D -i p1p1 -u snort -g snort -c
/etc/snort/snort.conf -l /var/log/snort -G 3


I suspect this message in the "messages" log folder (This is a CENTOS
box) may have been tied to the truncating error:

Jun 24 17:20:04 security1 rsyslogd-2177: imuxsock begins to drop messages
from pid 10279 due to rate-limiting


I started running Snort in the foreground as part of my testing.  Then I
received this error message:

 (snort_decoder) WARNING: IP dgm len > captured len


I performed binary captures at the same time while doing an Internet
download to compare results using TCPDUMP and Snort as follows (the
IP below is fake.  it represents my PC's address):

 snort -l ./logall -b -i p1p1 src 57.8.5.15 or dst 57.8.5.15

tcpdump -i p1p1 -N -w tcpdump.jun26.v1.pcap src 57.8.5.15 or dst 57.8.5.15

Doing a playback with snort -dvr showed clearly that the snort log
capture had a problem.  Joel recommended starting snort with an option to
force the capture length.  This then resulted in normal looking binary
captures and to my delight, an alert showed up when I tested an Internet
download (that a rule I have should detect)!  The fix was using -P on the
command line, here is how I start it now:

/usr/sbin/snort -A fast -d -D -P 9000 -i p1p1 -u snort -g snort -c
/etc/snort/snort.conf -l /var/log/snort -G 3


The default packet capture length is 1514, though for some reason I wasn't
even getting that.  This override has for the moment resolved my problem!!!
I would highly recommend starting Snort in the foreground on occasion just
to see if there are error messages you may not be aware of.    Now I can
finally get into doing the real work...Yeah!...oh no, I'm afraid of all the
work this is going to bring about.  Guess I'll be working late tonight!!!

Frank

On Wed, Jun 19, 2013 at 2:14 PM, James Lay <jlay () slave-tothe-box net> wrote:

> Hi again Frank.
>
> I would absolutely do the below:
>
>
> turn up some kind of sniffer on our CentOS box to see what is being passed
> to it on that interface, paying attention to the datagram sizes.
>
> If we have a hardware issue, nothing else is going to work well ;)  After
> verifying good hardware, then I'd do the repost to the mailing list.  I'd
> post the rules, the output of snort when you find one that slipped by (run
> it in the foreground then ctrl-c to get the info).  If you can get a small
> packet capture ready just in case one of the dev's wants it offline that
> would help.  Hope that works out Frank.
>
> James
>
>
> On 2013-06-19 10:54, Frank Calone wrote:
>
> > James,
> >
> >    I do have the full rules list as I have downloaded it and  have
> > pulled out some Java related ones to use on our Appliance with Snort
> > on it.  It already flagged one nice item!   In my testing I even
> > have used the exact same rule for the exe alerting on the appliance,
> > so I really dont see where doing anything with the rules will help me
> >
> > at this time.  The two rules I show below are very simple and just
> > looking for plain text.  I will look into the archive discussions to
> > see what I can find about logging.  I want to wait on that though
> > until I just get the alerting to work.  No need to worry about
> > logging if the alerts are being missed.  That is a good tip though
> > and I will look though the last couple of months archives.  So, at
> > this time you would agree I have these options:
> > 1) turn up some kind of sniffer on our CentOS box to see what is being
> > passed to it on that interface, paying attention to the datagram
> > sizes.
> > 2) contact the Snort development team asking for help.
> > 3) Repost this message again to the user list (how much of the
> > thread?).
> > 4) other options you may be thinking about????
> >
> > Frank
> >
> > On Wed, Jun 19, 2013 at 12:41 PM, James Lay <jlay () slave-tothe-box net
> > [5]> wrote:
> >
> > Ahhh...ok youre wanting to do a full packet capture once the rules
> > > fire...gotcha.  Three things I would do:
> > >
> > > Download both the latest snort rules and emerging threats rules,
> > > extract them into a dir, and grep for exe..there are a bunch of
> > > rules identifying executables and you can use those to your
> > > advantage and tweaking.
> > >
> > > Search the Snort mailing list archive to include last month...Im
> > >
> > > certain there was a discussion that hit upon full captures using
> > > snort when it fires on certain alerts.
> > >
> > > Lastly, if the mailing list archive doesnt give a solution, post the
> > > below info to the group (scrub/obfuscate the real IPs however)...the
> > > Sourcefire Engineers should have an answer on why this isnt working
> > >
> > > (it looks good to me and should be working correctly).
> > >
> > > Thanks Frank...hope this helps.
> > >
> > > James
> > >
> > > On 2013-06-19 10:07, Frank Calone wrote:
> > >
> > >  James,
> > > >    I do appreciate your thoughts on this.  The engine is very
> > > > current (2.9.4.5).  Im running CentOS and have 4 processors and a
> > > >
> > > >
> > > > total of 4 GB memory.  I have none of the distribution rules
> > > > loaded.  I am only running 2 rules looking for exes.  The
> > > > logging
> > > >
> > > > as I understand it was only to do packet captures for the session
> > > > that
> > > > was "tagged" due to an alert.     So, here is the startup
> > > > command:
> > > >
> > > > /usr/sbin/snort -A fast -d -D -i p1p1 -u snort -g snort -c
> > > > /etc/snort/snort.conf
> > > >  -l /var/log/snort2 -G 1
> > > >
> > > > Here are the 2 rules:
> > > >
> > > > alert tcp  $EXTERNAL_NET !20 -> $HOME_NET any (msg:"Snort_Windows
> > > > exe
> > > > down
> > > > loaded [not DOS mode] not recognized mirror"; content:"This
> > > > program
> > > > cannot be ru
> > > > n in DOS mode"; logto:"/var/log/snort/logto/**sid3000000.log";
> > > > flowbits:set,tagged
> > > >  ; tag:session,0,packets,1000,**seconds; sid:3000000; rev:5;)
> > > >
> > > > alert tcp $EXTERNAL_NET !20 -> $HOME_NET any (msg:"Snort_Win32 exe
> > > > downloa
> > > > ded [Win32 req]"; content:"This program must be run under Win32";
> > > > logto:"/var/lo
> > > > g/snort/logto/sid3000001.log"; flowbits:set,tagged;
> > > > tag:session,0,packets,1000,s
> > > >  econds; sid:3000001; rev:5;)
> > > >
> > > >
> > > > I have been unable to get the "logto" option to work.  I realize
> > > > the
> > > > manual says it will not work when the startup command is set to
> > > > binary
> > > > logging.  This is just a leftover from various testing I have
> > > > been
> > > > doing with the logging.  My understanding is that the flowbits
> > > > set
> > > > tagged   causes snort to say this session needs to be logged
> > > > when an
> > > > alert occurs.  The tab session, 0 , packets, 1000, seconds
> > > > is causing the session (all packets) to be captured in its
> > > > entirety
> > > > for up to 1000 seconds is necessary.  This has worked when we
> > > > get
> > > > hits, though on large binary captures (in the 50 MB sizes I am not
> > > > convinced I am getting everything.  I say this because the html
> > > > header info notes the "content size".  So I am able to determine
> > > > how
> > > > much was captured.  Smaller file sizes this has worked fine.
> > > > Larger
> > > > ones I usually dont get the exact value noted.  Im noting this
> > > > just
> > > >
> > > > so you can see what we are trying to do.  The bigger problem is
> > > > not
> > > > getting the alert at all.  As you can see our testing is very
> > > > simple
> > > > looking for plain text.
> > > >
> > > > Our network guy supposedly put a network sniffer on our tap last
> > > > night
> > > > and he said that the mirror looks to be working fine.  Full
> > > > packets
> > > > are showing up.  I suppose I could try to run wireshark on our
> > > > Snort
> > > > system to see what it sees coming in.  If there is a way to
> > > > enable
> > > >
> > > > the pcap component of Snort to do this, please do let me know how
> > > > you
> > > > would run it to see what is showing up to Snort.  Based on what
> > > > our
> > > > network guy has just told us, I am lead to believe there is a
> > > > problem
> > > > in the way our Snort server is handling the data stream.  I was
> > > > only
> > > > using 2 rules and yet it only occasionally flags the exes.  I was
> > > > not
> > > >
> > > > doing a binary capture on all of the packets, just the specific
> > > > sessions that alerted.  I feel like I am at wits end as to what
> > > > is
> > > > going wrong here.  I have performance monitoring turned on and
> > > > near
> > > > as I can see the system is usually 90% idle and memory usage
> > > > never
> > > > gets above 250 MB.  I setup stream5 memcap to 1 GB.  Im
> > > > scratching
> > > >
> > > > my head as to why then the datagram length is being truncated.  I
> > > > do
> > > > appreciate whatever suggestions you might have on this.  It seems
> > > > like what I am trying to do ought to work.
> > > >
> > > >  Yesterday I even removed the session tagging in the rule to just
> > > > get
> > > > the Snort application to alert on everything with no extra
> > > > logging
> > > > (see rules below).  I tried just 2 rules with one being even
> > > > simpler
> > > > (going to the youtube web site, I replaced my PCs IP address with
> > > > the
> > > >
> > > > 57.n.n.n one you see in the rule below for non-displosure
> > > > requirements).  Every time I refreshed my web page for youtube
> > > > the
> > > > alerts worked just fine.  When I downloaded putty.exe my new
> > > > Snort
> > > > server never alerted and yet our IBM appliance with Snort alerted
> > > > each
> > > > and every time.  Im ready to pull my hair out.
> > > >
> > > >
> > > > alert tcp EXTERNAL_NET any -> $HOME_NET any (msg:"Snort_DOS exe
> > > > downloaded [DOS req]"; content:"This program cannot be run in DOS
> > > > mode"; sid:3000001; rev:5;)
> > > >
> > > > alert tcp 57.198.5.15 any -> $EXTERNAL_NET any (msg:"Youtube";
> > > > content:"www.youtube.com [1] [1]"; sid:1999987;)
> > > >
> > > >
> > > >
> > > > Frank
> > > >
> > > >
> > > > On Wed, Jun 19, 2013 at 11:07 AM, James Lay
> > > > <jlay () slave-tothe-box net [2]
> > > > [2]> wrote:
> > > >
> > > > On 2013-06-19 08:44, Frank Calone wrote:
> > > > > James,
> > > > > >    Thanks for the tips.   I have looked through the
> > > > > > entire
> > > > > > Snort
> > > > > > manual.  Ive also used the site to search for specific
> > > > > > keywords
> > > > > > and
> > > > > >
> > > > > > such.  I look at all the user group postings now for well
> > > > > > over a
> > > > > > month (since we first tried using Snort).  I just am not a
> > > > > > pro
> > > > > > at
> > > > > > what you guys are doing.  The manual says Snort can do
> > > > > > binary
> > > > > > packet
> > > > > > capture of everything.  When I turned that on, that is when
> > > > > > I
> > > > > > saw the
> > > > > > problem (error message showing up).  I did not get that error
> > > > > > message
> > > > > > in the syslog when I was running snort the other way (as the
> > > > > > manual
> > > > > > recommends starting it).  Yet, this looks to explain well
> > > > > > the
> > > > > > randomness of my hits.  We have an appliance that monitors
> > > > > > web
> > > > > > as its
> > > > > > main purpose, but it also has a snort engine that we have been
> > > > > > using.  We look for exes and it does a good job of that.
> > > > > > The
> > > > > >
> > > > > > problem is it does not log at all.  We are wanting to log
> > > > > > so we
> > > > > > can
> > > > > > pull the exe out, then can do MD5,AV Totals, and other tests
> > > > > > to
> > > > > > determine if it is malicious.
> > > > > >    This other appliance then tells me what I ought to be
> > > > > > alerting
> > > > > > on.  Yet, I only get an occasional hit.  If the packets are
> > > > > > not
> > > > > > complete with payload (datagram issue), that would explain my
> > > > > > problem.  I am not convinced our network tap is giving me
> > > > > > everything,
> > > > > > though my boss thinks it is.  So, when I turned up full
> > > > > > capture
> > > > > > yesterday with Snort, I could see I was mostly just getting
> > > > > > the
> > > > > > header.  What I dont know here is whether Snort itself is
> > > > > > truncating
> > > > > >
> > > > > > the packets or if it is the network appliance that is giving
> > > > > > us
> > > > > > our
> > > > > > tap.  I really think this is the heart of my issue and I need
> > > > > > to
> > > > > > figure out which is the problem.  So when you suggest a pcap
> > > > > > capture
> > > > > > I am presuming you are suggesting using a networking appliance
> > > > > > or
> > > > > > other tool to see what I am getting on the wire correct?
> > > > > > There
> > > > > > isnt
> > > > > >
> > > > > > some other setting within Snort that I need to be using to do
> > > > > > this is
> > > > > > there?
> > > > >
> > > > > Hi Frank,
> > > > >
> > > > > I can totally see where doing a full packet capture using snort
> > > > > would cause issues..its busy logging packets instead of doing
> > > > > its
> > > > >
> > > > > IDS job.  I would suggest using tcpdump or tshark to do a full
> > > > > packet capture off the tap (maybe just plug in a laptop running
> > > > > linux and either of the above) for an hour and see if you get
> > > > > full
> > > > > data.  Id also recommend making sure the Snort engine is up to
> > > > > date
> > > > >
> > > > > as well as the rulesets.  I would not use Snort as a full
> > > > > packet
> > > > > capture plus IDS at the same time.  Hope that helps.
> > > > >
> > > > > James
> > > >
> > > > Links:
> > > > ------
> > > > [1] http://www.youtube.com/ [3]
> > > > [2] mailto:jlay@slave-tothe-box.**net <jlay () slave-tothe-box net> [4]
> >
> >
> > Links:
> > ------
> > [1] http://www.youtube.com/
> > [2] mailto:jlay@slave-tothe-box.**net <jlay () slave-tothe-box net>
> > [3] http://www.youtube.com/
> > [4] mailto:jlay@slave-tothe-box.**net <jlay () slave-tothe-box net>
> > [5] mailto:jlay@slave-tothe-box.**net <jlay () slave-tothe-box net>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!