Re: [Snort-devel] number of packets processed is mis-calculated under Linux

[Phil Wood <cpw () lanl gov>]

Darn!  That's a fix I made to libpcap which I maintain that uses a 
shared memory ring.  I forgot all about it and thought it was available
from tcpdump.org.

I apologize for the error.  The libpcap that I maintain tracks the
current libpcap at tcpdump.org.  It's major change has to do with using
shared memory that supports a ring buffer shared between the user
application and the kernel.  A secondary effort was made to capture
statistics both from the kernel and the application such that one could
determine when and where packets were being dropped.  If you are interested,
see:

  http://public.lanl.gov/cpw

Sorry,

On Mon, Dec 08, 2003 at 06:13:18PM +0100, Bart Haagdorens wrote:
> Hi,
>
> I skipped over your text briefly, and in my opinion it is largely correct.
> It seems to support my interpretation of the 'recv' and 'drop' variables,
> which is different from snort's interpration.
>
> I don't agree on this one sentence however:
>
> The linux libpcap subroutine pcap_stats, which calls this system call,
> accumulates them so that each call to it returns the grand total since
> initializing the ring.
>
> Can you support this claim? I read the 0.7.2 libpcap pcap-linux.c sources,
> and found no such "grand totalling" in pcap_stats. To me, it would seem that
> the system call results are simply returned without any totalling.
>
> Regards,
>
> Bart Haagdorens
>
>
> -----Original Message-----
> From: Phil Wood [mailto:cpw () lanl gov] 
> Sent: maandag 8 december 2003 17:58
> To: Bart Haagdorens
> Cc: Snort-devel () lists sourceforge net
> Subject: Re: [Snort-devel] number of packets processed is mis-calculated
> under Linux
>
>
> FYI,
>
> If you are running with a bpf filter (on linux) your received count will be
> wrong.  On linux filtering is accomplished prior to bumping the received 
> count.  Snort should probably keep it's own received (processed) count, as 
> tcpdump does, since there could be a number of packets that snort has not
> processed when it is asked to dump some stats.
>
> BSD on the other hand filters after bumping the recv'd count.
>
> By rights BSD and Linux stats are comparable only when no filter is applied
> and you are running a version of libpcap on linux that maintains running
> totals.  This is because the kernel system call for PACKET_STATISTICS on
> linux resets the the stats to zero after returning the results.  The BSD
> kernel maintains a running total for you.  
> A libpcap application should probably watch out for when these totals
> rollover.
>
> Let me know if my cut on the subject at:
>
>   http://public.lanl.gov/cpw/packet_stats
>
> has any errors, or could be improved.
>
> Thanks,
>
> On Mon, Dec 08, 2003 at 12:27:35PM +0100, Bart Haagdorens wrote:
> > Hi,
> >
> > I want to report a bug in snort's statistics display when processing 
> > 'live' traffic on a Linux platform.
> >
> > The problem is due to the interpretation of ps_recv and ps_drop 
> > returned by pcap-stats. In the libpcap 0.7.2 sourcecode for Linux 
> > (pcap-linux.c), line 664, an interesting comment describes the correct 
> > interpretation of theses
> > numbers:
> >
> >  ps_recv "counts all packets handed to the PF_PACKET socket, including 
> > packets dropped because there wasn't room on the socket buffer (...)"  
> > ps_drop "is incremented for every packet dropped because there's not 
> > enough free space in the socket buffer."
> >
> > So, the total number of packets that passes the filter is (ps_recv), 
> > the number of packets dropped is (ps_drop) and the number of packets 
> > that is put into the socket buffer is (ps_recv-ps_drop). We verified 
> > this interpretation with some experiments on live traffic.
> >
> > The interpretation in snort 2.0.5, util.c, line 917 is clearly wrong:
> >
> >             LogMessage("Snort analyzed %d out of %d packets, ", 
> >                     ps.ps_recv, ps.ps_recv+ps.ps_drop);
> >
> >             if(ps.ps_recv)
> >             {
> >                 LogMessage("dropping %d(%.3f%%) packets\n\n", 
> >                         ps.ps_drop, 
> >                         CalcPct( (float) ps.ps_drop, (float)
> > (ps.ps_recv+ps.ps_drop) ));
> >             }
> >
> > The number of dropped packets is already included in ps_recv, but it 
> > is added once again! So the statistics count the dropped packets 
> > twice, which can be very confusing.
> >
> > Correct would be:
> >
> >             LogMessage("Snort analyzed %d out of %d packets, ", 
> >                     ps.ps_recv-ps.ps_drop, ps.ps_recv);
> >
> >             if(ps.ps_recv)
> >             {
> >                 LogMessage("dropping %d(%.3f%%) packets\n\n", 
> >                         ps.ps_drop, 
> >                         CalcPct( (float) ps.ps_drop, (float) 
> > (ps.ps_recv) ));
> >             }
> >
> > I don't know how the numbers should be handled on other platforms, but 
> > for Linux the current interpretation is clearly mistaken.
> >
> > Regards,
> >
> > Bart Haagdorens
> >
> > _________________________________
> > Bart Haagdorens
> > Vrije Universiteit Brussel
> > TW-INFO
> > Pleinlaan 2
> > B-1050 Brussels
> > Belgium
> >
> > bart.haagdorens () vub ac be
> >
> > Tel: +32 2 629 24 94
> > GSM: +32 477 69 77 21
> >
> >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: IBM Linux Tutorials. Become an 
> > expert in LINUX or just sharpen your skills.  Sign up for IBM's Free 
> > Linux Tutorials.  Learn everything from the bash shell to sys admin. 
> > Click now! http://ads.osdn.com/?ad_id78&alloc_id371&op?k
> > _______________________________________________
> > Snort-devel mailing list
> > Snort-devel () lists sourceforge net 
> > https://lists.sourceforge.net/lists/listinfo/snort-devel
>
> -- 
> Phil Wood (cpw_at_lanl.gov)

-- 
Phil Wood (cpw_at_lanl.gov)


-------------------------------------------------------
This SF.net email is sponsored by: IBM Linux Tutorials.
Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
Click now! http://ads.osdn.com/?ad_id78&alloc_id371&op=click
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users