Re: Bugs in Packet I/O Totals section

[elof2 () sentor se]

This is just an update to inform you that snort version 2.9.7.2 (Build 
177) still has this problem.


snort[50090]: *** Caught Term-Signal
snort[50090]: 
===============================================================================
snort[50090]: Run time for packet processing was 55872.623158 seconds
snort[50090]: Snort processed 4904147604 packets.
snort[50090]: Snort ran for 0 days 15 hours 31 minutes 12 seconds
snort[50090]:     Pkts/hr:    326943173
snort[50090]:    Pkts/min:      5267612
snort[50090]:    Pkts/sec:        87774
snort[50090]: 
===============================================================================
snort[50090]: Packet I/O Totals:
snort[50090]:    Received:    629464645
snort[50090]:    Analyzed:   4904147604 (779.098%)
snort[50090]:     Dropped:     24335630 (  3.722%)
snort[50090]:    Filtered:            0 (  0.000%)
snort[50090]: Outstanding:            0 (  0.000%)




Error #1:
The amount of Outstanding packets shouldn't be 0 (there should be lots of 
them on my choked test-machine).

Error #2:
Received is too low.

Error #3:
Percentages are wrong, but this is due to error #2.

/Elof


On Thu, 17 Jul 2014, elof2 () sentor se wrote:

> When I send an USR1 signal to the snort process every 10 minutes, I get
> sane counter values in the dumped stats, all the time.
> When I then kill (HUP) the process, the exit stats also look sane.
>
> Examples after snort has been running for 23 hours:
>
> USR1:
> *** Caught Dump Stats-Signal
> ===============================================================================
> Packet I/O Totals:
>    Received:   8717355239
>    Analyzed:   7184525494 ( 82.416%)
>     Dropped:   1528109289 ( 14.915%)
>    Filtered:            0 (  0.000%)
> Outstanding:   1532829745 ( 17.584%)
>    Injected:            0
> ===============================================================================
>
> (the test-machine I'm running on is choked so the capture drops and
> outstanding packets are ok)
>
> Everything's looking good.
>
>
> Now I HUP the process:
> *** Caught Term-Signal
> ===============================================================================
> Run time for packet processing was 86339.339581 seconds
> Snort processed 7184545604 packets.
> Snort ran for 0 days 23 hours 58 minutes 59 seconds
>     Pkts/hr:    312371548
>    Pkts/min:      4996206
>    Pkts/sec:        83213
> ===============================================================================
> Packet I/O Totals:
>    Received:   8717412692
>    Analyzed:   7184545604 ( 82.416%)
>     Dropped:   1528109289 ( 14.915%)
>    Filtered:            0 (  0.000%)
> Outstanding:   1532867088 ( 17.584%)
>    Injected:            0
> ===============================================================================
>
> Still everything's looking good.
>
>
>
>
> However...
> If I don't send any USR1 signal to the snort process at all, but
> instead HUP it after several hours, then the exit stats are messed up:
>
> HUP:
> *** Caught Term-Signal
> ===============================================================================
> Run time for packet processing was 51301.212425 seconds
> Snort processed 4096191308 packets.
> Snort ran for 0 days 14 hours 15 minutes 1 seconds
>     Pkts/hr:    292585093
>    Pkts/min:      4790867
>    Pkts/sec:        79846
> ===============================================================================
> Packet I/O Totals:
>    Received:    466634848
>    Analyzed:   4096191308 (877.815%)
>     Dropped:    660204936 ( 58.589%)
>    Filtered:            0 (  0.000%)
> Outstanding:            0 (  0.000%)
>    Injected:            0
> ===============================================================================
>
>
>
> Snort processed 4096191308 packets.
>     Analyzed:   4096191308
>
> These two lines look sane, but what about Received?
>
>     Received:    466634848
>
> This value is way too small!
> Also, there's suddenly no Outstanding value at all!
> = Two errors.
>
> (The wonky percentage of 877.815% is a side-effect of the too small
> Received value)
>
>
>
> So what is happening here?
>
> I can't say for sure, but I think the problem manifest itself when
> Received gets above 2^32 (4 294 967 296) packets and no USR1 signal has
> been sent.
> Because if I HUP the process after only a few minutes, the stats always
> look sane and correct.
>
> I just don't understand how Received and Outstanding can be correct (and
> with numbers larger than 2^32) as long as an USR1 signal is sent or as
> long as the process hasn't been running for long.
> (...and without an answer to my initial four questions in my first email
> (below), I don't know where the values come from to begin with, so it is
> harder to draw conclusions.)
>
>
> note: I also ran 'netstat -B' in FreeBSD every 10 minutes and its
> bpf-stats values look sane, and they correspond nicely with the sane
> values from snort USR1, so I don't believe the wrong value is coming from
> the bpf stats in the FreeBSD operating system but from the DAQ subsystem.
>
>
> All of this is reproduceable every day.
>
>
>
> I'm running:
> FreeBSD 10.0 amd64
> Snort Version 2.9.6.1 (Build 56)
> Using libpcap version 1.4.0
> Using DAQ module pcap(v3)
>
>
> /Elof
>
>
>
>
> On Wed, 16 Jul 2014, elof () sentor se wrote:
>
> > When stopping snort, or dumping stats, you get this section:
> >
> > ===============================================================================
> > Packet I/O Totals:
> >    Received:   wwwwwww
> >    Analyzed:   xxxxxxx ( 99.811%)
> >     Dropped:   yyyyyyy (  0.730%)
> >    Filtered:         0 (  0.000%)
> > Outstanding:   zzzzzzz (  0.189%)
> >    Injected:         0
> > ===============================================================================
> >
> > Filtered is not supported by the pcap DAQ, so 0.
> > Injected is 0 since I'm not running in inline mode.
> >
> > No questions about these two. But...
> >
> >
> > 1) Exactly where is the Received value coming from?
> > Is it an internal counter of *actually received packets* within snort, or
> > is this value supplied by the daq-system, bpf-system or simillar?
> >
> > 2) I guess analyzed is the amount of packets from the received ones that
> > actually made it all the way through snort processing. Correct? ...or is
> > this aquired elsewhere?
> >
> > 3) Dropped seem to be the reported drop count from the bpf-system. This
> > should mean that Dropped = "Capture drops (drops outside of snort)".
> > Correct?
> >
> > 4) Outstanding seem to simply be Received minus Analyzed. Correct?
> >
> >
> >
> > I get very confusing numbers, that's why I'm asking.
> > When I have descriptions of what the values should be, I can create a
> > future bug report, if needed.
> >
> >
> > So, for the four titles above, can I have a short description of what they
> > truly are and where the values come from?
> >
> > /Elof
> >
> > ------------------------------------------------------------------------------
> > Want fast and easy access to all the code in your enterprise? Index and
> > search up to 200,000 lines of code with a free copy of Black Duck
> > Code Sight - the same software that powers the world's largest code
> > search on Ohloh, the Black Duck Open Hub! Try it now.
> > http://p.sf.net/sfu/bds
> > _______________________________________________
> > Snort-devel mailing list
> > Snort-devel () lists sourceforge net
> > https://lists.sourceforge.net/lists/listinfo/snort-devel
> > Archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
> >
> > Please visit http://blog.snort.org for the latest news about Snort!
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Snort-devel mailing list
> Snort-devel () lists sourceforge net
> https://lists.sourceforge.net/lists/listinfo/snort-devel
> Archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>
> Please visit http://blog.snort.org for the latest news about Snort!

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!