Re: Libpcap on VMWare

[Guy Harris <guy () alum mit edu>]

On Jan 12, 2010, at 5:59 PM, Mark Bednarczyk wrote:

> No drops on NON-vmware platforms.

So at least some of the problem could involve a code path difference on VMware, e.g. either the VMware code itself or 
whatever code in the guest OS receives packets from VMware.  Is VMware simulating a hardware device, with Linux just 
running the regular driver for that simulated hardware, or is, for example, some amount of paravirtualization being 
done?

> I can comment about the way that pcap_dispatch vs. pcap_loop seem to behave.
> I have tried my tests with both functions and both drop packets. What is
> surprising even at high packet rates is that pcap_dispatch does not seem to
> buffer more that 1 or 2 packets before exiting the call. I would think that
> at a higher packet rate, libpcap would like to buffer as many packets as the
> ring-buffer allows before exiting pcap_dispatch call.

At least on Ubuntu, he's using libpcap 0.9.8; unless it's a modified 0.9.8, it's not using the ring buffer - the 
receive loop just reads packets on at a time with recvbuf(), and exits once it's accepted a packet.  In that case, I'd 
be surprised if it *ever* delivered more than one packet.

Libpcap 1.0.0 and later should use the ring buffer and deliver more packets per call, as per my and Dustin Spicuzza's 
mail.

> This means that in our
> test application, our outside loop around pcap_dispatch call has to call on
> pcap_dispatch frequently since very few packets are processed per loop
> iteration.
>
> When testing with pcap_loop, the behaviour is a bit more as expected.

pcap_loop() repeatedly calls the "read a bufferfull of packets" routine until the count is exhausted (or forever if a 
count of -1 is supplied), an error occurs, or pcap_breakloop() is called.  pcap_dispatch() calls it only once; in fact, 
it's

        int
        pcap_dispatch(pcap_t *p, int cnt, pcap_handler callback, u_char *user)  
        {
                return p->read_op(p, cnt, callback, user);
        }

in modern versions of libpcap ("modern" here includes the 0.9.x series), and, given that pcap_loop() is

        int
        pcap_loop(pcap_t *p, int cnt, pcap_handler callback, u_char *user)
        {
                register int n;

                for (;;) {
                        if (p->sf.rfile != NULL) {
                                /*
                                 * 0 means EOF, so don't loop if we get 0.
                                 */
                                n = pcap_offline_read(p, cnt, callback, user);
                        } else {
                                /*
                                 * XXX keep reading until we get something
                                 * (or an error occurs)
                                 */
                                do {
                                        n = p->read_op(p, cnt, callback, user);
                                } while (n == 0);
                        }
                        if (n <= 0)
                                return (n);
                        if (cnt > 0) {
                                cnt -= n;
                                if (cnt <= 0)
                                        return (0);
                        }
                }
        }

pcap_loop(), on a live capture, could be thought of as equivalent to

        int
        pcap_loop(pcap_t *p, int cnt, pcap_handler callback, u_char *user)
        {
                register int n;

                for (;;) {
                        /*
                         * XXX keep reading until we get something
                         * (or an error occurs)
                         */
                        do {
                                n = pcap_dispatch(p, cnt, callback, user);
                        } while (n == 0);
                }
                if (n <= 0)
                        return (n);
                if (cnt > 0) {
                        cnt -= n;
                        if (cnt <= 0)
                                return (0);
                }
        }

so the only reason why you'd get more calls to the callback from pcap_loop() than pcap_dispatch() would be that 
pcap_loop() is like a routine that repeatedly calls pcap_dispatch() until it gets "enough" packets.

> I have mapped out where libpcap pcap_stats indicates packet drops in a loop around
> the pcap_loop call. That is: capture 10K packet 10 times. Consitently the
> packet drops appear (as reported by pcap_stats) within the first 1K packet
> batch captured with the new pcap_loop call. Once the first 1000 packets have
> been dispatched there doesn't appear to be any more drops until the
> pcap_loop call exits and is re-called again for another 10K packets. The
> same test using pcap_dispatch points at packets being dropped all over. It
> seems that packet drops occur shortly in between consecutive
> pcap_dispatch/pcap_loop calls. Atleast in pcap_loop case there seems to be
> some catching up going on, and then things stabilize. Since pcap_dispatch
> exits much more frequently then pcap_loop that packet drops appear more
> frequently through out the test loops.

The only difference between pcap_dispatch() and pcap_loop() should be one extra user-mode procedure call per packet 
when calling pcap_dispatch(); perhaps that's sufficient to make a difference (even with the extra tests done in 
pcap_loop()), but it's not as if the system call pattern would be different between pcap_dispatch() and pcap_loop().

Presumably you're calling pcap_stats() for *every* packet, so it should report comparable information for 
pcap_dispatch() and pcap_loop() (calling it after each pcap_dispatch() or pcap_loop() call will obviously show that the 
packet drops occur between consecutive pcap_dispatch() or pcap_loop() calls :-)).

-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.