Re: advice for heavy traffic capturing

["Fulvio Risso" <fulvio.risso () polito it>]

Hi Darren.

> -----Original Message-----
> From: tcpdump-workers-owner () lists tcpdump org
> [mailto:tcpdump-workers-owner () lists tcpdump org]On Behalf Of Darren Reed
> Sent: domenica 8 agosto 2004 17.09
> To: tcpdump-workers () lists tcpdump org
> Subject: Re: [tcpdump-workers] advice for heavy traffic capturing
>
>
> In some email I received from Fulvio Risso, sie wrote:
> > Darren, could you please give us some numbers?
> > If you take a look at this paper:
> >
> >   F. Risso, L. Degioanni
> >   An architecture for high performance network analysis
> http://ieeexplore.ieee.org/iel5/7446/20240/00935450.pdf?tp=&arnumb
> er=935450&
> >
> isnumber=20240&arSt=686&ared=693&arAuthor=Risso%2C+F.%3B+Degioanni
> %2C+L.%3B
>
> I don't have an IEEE login.  Feel free to email it to me.

  http://netgroup.polito.it/fulvio.risso/pubs/iscc01-wpcap.pdf


> > and this:
> >
> >   L. Deri
> >   Improving Passive Packet Capture:Beyond Device Polling
> >   http://luca.ntop.org/Ring.pdf
>
> I tried the patches, for Linux, out from this paper and they hung the
> machine.  I didn't have time to play so I moved on.  This could very
> well be to do with the problem of 'N' versions of the Linux kernel and
> the one I tried was not the exact same one as the people who did the
> development.  I don't know and I don't care - it didn't work for me.
>
> > it seems that Windows is the most performing OS (without any
> ad-hoc patch).
>
> Maybe it is more to do with drivers and NICs or maybe not.  I was able to
> get similar performance out of FreeBSD and NetBSD (on the same hardware),
> without any tweaks.  Maybe {Free,Net}BSD have better drivers for Intel
> hardware than Linux ?  *shrug*

Hardware counts, but... we have been really careful to optimize the whole
path from the NIC card to the application.
See another article on this topic (it covers only Win32):

   L. Degioanni, M. Baldi, F. Risso, G. Varenni
   Profiling and Optimization of Software-based Network Analysis
Applications
   http://netgroup.polito.it/fulvio.risso/pubs/sbac03-winpcap.pdf

Particularly, Figure 9 shows how much work has been done to reduce the
processing overhead.
And yes, NIC drivers and OS overheads are very important... but these are
the components that cannot be changed by normal users.


> > Do you have anything (possible published somewhere) supporting
> what you're
> > saying?
>
> No...I was going to do something like that but it requires permission from
> the people I'm doing the work for to do so.

This should be very very helpful.


> But, to give you an idea of the sort of problems we saw with Linux, moving
> from RedHat 9 (kernel 2.4) to Fedora (kernel 2.6), we saw an increase in
> packet loss of 10x (one order of magnitude) from as low as 1Mbps all the
> way up.  When we saw that we realised that it was time to
> starting planning
> to move away from Linux.

In my personal opinion (very very personal), get rid off Linux for packet
capture.
Use *BSD instead, if you want to use Unix.

        fulvio

-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.