Re: bpf/pcap performance

[Guy Harris <guy () alum mit edu>]

(Noise inserted in the hopes that that the mailing list software doesn't
think that this is a duplicate of my previous message, which I sent from
my sonic.net address and which thus didn't get through, and thus prevent
it from getting to the list.)

On Wed, Apr 14, 2004 at 12:30:45PM +1000, Darren Reed wrote:
> > > Looking more closely at the non-blocking side of things, there seems
> > > to be no use of either p->nonblock or p->timeout apart from in
> > > pcap_setnonblock() ?
> >
> > "p->nonblock" is there for the benefit of, among other things, 
> > "pcap-linux.c" if, as, and when it implements mmaped-capture support - 
>
> Ok, well:
> fgrep '>nonblock' *.c
> pcap-win32.c:   return (p->nonblock);
> pcap-win32.c:   p->nonblock = (newtimeout == -1);
> pcap.c: * We don't look at "p->nonblock", in case somebody tweaked the FD
>
> What happened to linux ?

It hasn't had the mmaped capture support implemented yet in the
tcpdump.org CVS.

> I might add that ethereal could really benefit from something like
> this (perhaps).  It currently behaves very poorly when it comes to
> dealing with large capture files (large might be as small as 100MB
> or even less.)  Unless this is just a problem isolated to using it
> on windows ?  And yes, when you're capturing gigabytes per day,
> this is a small amount :)

There are plenty of other per-packet-read-in CPU-time-chewers in
Ethereal; once those are changed, perhaps the time spent copying the
data around will show up.

> Something like map the file from disk and just build an internal
> index, rather than try read the entire file in (which is what it
> appears to do.)

Yes, it does, indeed, read the entire file, because it needs to dissect
every packet in order to

        1) create the summary list items for those packets

and

        2) construct all the state that might be necessary to dissect
           many of the packets.

> > > I think the current design (pcap_dispatch(), etc) is what's led
> > > to strange API hooks like pcap_breakloop() appearing, when I think
> > > a better design could have side stepped the need for them O:-)
> >
> > Well, "pcap_breakloop()" also works around the workaround :-) for the 
> > "don't choke when we get ptraced" problem, i.e. that the implementation 
> > on some platforms treats EINTR not as an indication that it should quit 
> > but as an indication that it should try reading again.
>
> read(2) can also return EINTR if a signal is received, depending on how
> signal handling was setup by the application.  e.g. SIGHUP.

Yes, that's what I was referring to - if you've set up signal handling
to interrupt system calls, then after a signal handler returns, EINTR
will be returned, but the read loop will go back and read more data
rather than returning.

> > I don't know 
> > whether an EINTR gets delivered if you're debugging a libpcap-based 
> > application, but, if that *is* a problem and that workaround *is* 
> > necessary, some scheme is needed to let a signal handler cause the read 
> > loop to terminate.
>
> For example, ^C'ing tcpdump, having it break out of the pcap_dispatch()
> loop and exiting back out so that it can do a pcap_dump_flush() and
> exit cleanly, without any data lost to internal buffers, right ?

Yes, that's the idea.
-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.