Re: questions on -B, performance, mbufs, and

[Jon Schipp <jonschipp () gmail com>]

Hello Guy,

Thanks for the response, you've helped me a great deal.

I increased the libpcap buffer size by incrementing the value in
net.bpf.size/maxsize.

Test1: capture 24 hours with no buffer or OS changes(bpf size = 4096, max
size = 524288), tcpdump with -nn -w -s0
Result: Captured 47,086,090 - Recv by filter 47,196,268 - Dropped by kernel
110,168

Test2: capture 24 hours, increased the bpf buffers and therefore the libpcap
buffer (bufsize = 524288, maxbufsize = 1,048,576), tcpdump with -nn -w -s0
(all same)
Result: Captured 48,711,832 - Recv by filter 48,718,044 - Dropped by kernel
5,744

A significant improvement!

I want to try and increase it further and test the results, say, by 1MB  or
512K more. I'm trying to use the -B option this time but it's not working
for me, tcpdump just prints the help menu.
tcpdump -B 2000000 -nni em0 -s0 -w test.txt
I've tried other values as well e.g. 524288, 1048576. It doesn't like any of
them.  *I'm on FreeBSD 8.2-Release and I just compiled 4.1 from the ports
though it still reports 4.0 in the help, (system came with 4.0)* What could
I be doing wrong? Feature not available in FBSD?

One last question,  I had a capture that reported:
captured 92,505,041
recvfilter 92,505,061
dropped  0

There's a difference of 20 packets between the captured and recv filter
numbers. If the kernel didn't drop any, that must mean that tcpdump couldn't
hand 20 of them, right?

Thanks!

On Wed, Sep 14, 2011 at 2:53 AM, Guy Harris <guy () alum mit edu> wrote:

> On Sep 13, 2011, at 8:47 PM, Jon Schipp wrote:
>
> > I have a few questions, hopefully someone can set me straight.
> > Info: *I'm on FreeBSD 8.2* *out of a couple million packets, in a few
> hours
> > time, I drop around 4000*
> > First off are all packets stored in mbufs?
>
> With most network drivers, I think packets are stored in the driver's ring
> buffer, and those are "loaned" to mbufs.  Or maybe I'm remembering SunOS
> 4.x, and FreeBSD does things differently.
>
> > And if so, is it possible to increase the amount of mbuf clusters to
> improve
> > tcpdump performance?
>
> That's probably not the problem.  The mbuf or mbuf chain containing an
> input packet is handed to BPF if you're capturing traffic with libpcap
> (which tcpdump does), and they're copied to BPF's own buffer, which is not
> made of mbufs.
>
> > Again, questioning myself, I wasn't sure if packets for tcpdump were
> stored
> > in mbufs, I would expect them to run out if the kernel was dropping them.
> > Though, I'm assuming that all packets are stored in mbufs and that bpf
> just
> > gets a copy of what's in the mbufs.
>
> They are, but the buffer into which the copy is made is not mbufs, it's set
> on a BPF device with a BIOCSBLEN ioctl.  ("man bpf".)
>
> > Second, does -B set an application buffer independent of the libpcap
> buffer?
>
> No, it tells libpcap how big the buffer size should be; in BPF (*BSD and
> Mac OS X), that's the kernel buffer size as set by BIOCSBLEN.
>
> > What's the default size?
>
> In the current top-of-tree libpcap, 524288 bytes.
>
> In FreeBSD 8.2, 32768 bytes.
>
> > Can capture improvements be made by increasing this value?
>
> Probably.  32768 may have made sense back when a machine with 32 megabytes
> was a Big Machine With Tons Of Memory and 100 MB/sec Ethernet was a Really
> Fast Network, but the laptop on which I'm typing this has 4GB of memory
> (when I bought it, that was twice the base memory size; it now *is* the base
> memory size) and its Ethernet adapter is a 10/100/1000 interface.  (It was
> boosted in response to a gripe from a networking developer in the OS group
> for the machine in question, because those of us in the remote file system
> group wanted tcpdump to default to 65535 as the snapshot length, which
> caused packet drops with the old default buffer size.  I took one look at
> the default buffer size and said "hello, 1998 called, it wants its default
> buffer size back" - although maybe I'm being unfair to 1998.)
>
> > *I'm dropping packets at the kernel, according to tcpdump*
> > Is the argument for -B in Kilobits?
>
> No, it's in bytes.
>
> > I see that the libpcap buffer can be increased with sysctl since libpcap
> > initializes its buffer amount with bpf sizes:
> > <
> majordomo () lists tcpdump org?subject=subscribe%20to%20tcpdump-workers&body=subscribe%20tcpdump-workers
> > net.bpf.maxbufsize:
>
> That's the *maximum* buffer size; you can try to set a bigger buffer with
> "-B" (or with pcap_set_buffer_size() in libpcap, which tcpdump calls to
> implement "-B", or with BIOCSBLEN, which libpcap does with the buffer size
> from pcap_set_buffer_size()), but the kernel will silently truncate that
> size at net.bpf.maxbufsize.
>
> > net.bpf.bufsize:
>
> That's the default, which you *can* set bigger, which will set the buffer
> size if you don't specify "-B", but it'll also set the buffer size used by
> any application or daemon that doesn't explicitly set it.  I don't know what
> else used BPF on FreeBSD, but in Mac OS X, the DHCP client uses it, and when
> I set it (and the maximum size) to a very large value (on a machine older
> and smaller than the machine on which I'm typing this) that meant I ran out
> of memory when running tcpdump, so the DHCP client couldn't re-acquire the
> machine's configuration when it woke up, leaving it unable to access the
> network.  So if you crank net.bpf.bufsize up, be careful....
>
> > I plan on doing tests with these soon. I'm trying to collect as much
> > information as possible first.
>
> I'd consider using "-B 524288" rather than using sysctl to set either
> net.buf.maxbufsize or net.bpf.bufsize.  *If* 524288 isn't big enough, crank
> net.bpf.maxbufsize up and pick a bigger value to pass to "-B".
>
> > Third, when tcpdump reports how many packets are dropped by the kernel is
> > this the same value as the packets dropped by libpcap?
>
> It's "ps_drop" from the result of pcap_stats(), which is "bs_drop" from
> BIOCGSTATS on systems using BPF, which is the number of packets that the BPF
> code in the kernel dropped because there wasn't enough space in the BPF
> buffer.
>
> > In otherwords, does libpcap ask the kernel for the kernel drop amount
>
> Yes.
>
> > and report it as "packets dropped by kernel".
>
> It reports it as "ps_drop", which tcpdump reports as "packets dropped by
> kernel".
>
> > Or will they have different
> > values?
> > The reason I ask is that the program ntop has a field that shows "dropped
> by
> > (libpcap)" but not a "dropped by kernel"
>
> Hmm.  I'm looking at the ntop 4.1.0 source, and can't find "dropped by
> (libpcap)" in a message.  There's "STATS: {N} packets dropped (according to
> libpcap)", which is reporting ps_drop from pcap_stats(), and there's
> "Dropped (libpcap)" and "Dropped (pcap)" in some HTML-generating code, which
> is also reporting ps_drop from pcap_stats().
>
> So ntop is saying "libpcap" where tcpdump is saying "kernel", but they're
> reporting the same value.
>
> > If they're different, is there a way to find out the root cause: kernel
> or
> > libpcap
>
> libpcap doesn't itself have any buffer that can overflow - it's all kernel.
>
> -
> This is the tcpdump-workers list.
> Visit https://cod.sandelman.ca/ to unsubscribe.



-- 
- Jon
-- 
------------------------------------------------------------------

VMB: 812-682-0231

Dubois County Linux User Group - http://www.dclinux.org
Southern Indiana Computer Klub - http://sickbits.networklabs.org
Bloomington FOOLS - http://www.bloomingtonfools.org/
BloomingLabs -  http://www.bloominglabs.org
ISSA-Kentuckiana  -  http://issa-kentuckiana.org

GPG Key ID: 810903CB
Key fingerprint = 0069 ED69 EABB DF84 5983  AD3C 6C20 BEFD 8109 03CB
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.