Re: advice for heavy traffic capturing

[Darren Reed <darrenr () reed wattle id au>]

[ Charset ISO-8859-1 unsupported, converting... ]
>   http://netgroup.polito.it/fulvio.risso/pubs/iscc01-wpcap.pdf

When was it published?  There is no date...

Winpcap appears, by design, to be the same as BPF.  If you reduced the
number of buffers in the ring used with NPF to 2 buffers, I suspect it
would be the same as BPF ?

And because there is no date, I can say that references to the buffer
size being 32Kbytes in recent BSD kernels is wrong.  Recent BSD kernels
use 1MB or 2MB buffers, by default.  Although it then contradicts itself
later by saying there are larger buffers but that pcap tunes it down to
32K....(page 2 vs page 3.)

> Hardware counts, but... we have been really careful to optimize the whole
> path from the NIC card to the application.
> See another article on this topic (it covers only Win32):
>
>    L. Degioanni, M. Baldi, F. Risso, G. Varenni
>    Profiling and Optimization of Software-based Network Analysis
> Applications
>    http://netgroup.polito.it/fulvio.risso/pubs/sbac03-winpcap.pdf

No date on the paper, here, either.

> Particularly, Figure 9 shows how much work has been done to reduce the
> processing overhead.

Interestingly, there are a few large areas for improvement: timestamp
(1800 -> 270), Tap processing (830->560) and filtering (585 -> 109).

> And yes, NIC drivers and OS overheads are very important... but these are
> the components that cannot be changed by normal users.

I think that's what you're seeing with the 3Com GigE NIC for 100BT
receiving.  Do you know what size the buffers on the card are ?

The Intel 100 ProS have 128K for receieve, as I recall, the same as
the 1000MX card.  There wasn't much between these two, that I was able
to observe, except that the 100ProS was slightly better.

My biggest problem here is that you've expended effort to tune and make
NPF fast (which is fine) and compare it with existing BPF, almost to say
that BPF is bad.  I suppose this is what researchers do, but I think it
is unfair on BPF.  IMHO, you should have tested with the same buffer size
for both, even if it meant hacking on libpcap. 

In the NetBSD emails, I think I ponder making changes to the buffering
so that it is more ring-buffer like (similar to what exists within NPF
if I understand the diagram right.)

Is the JIT code easily ported to other platforms ?

Darren
-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.