netmap support for libpcap now available

[Luigi Rizzo <rizzo () iet unipi it>]

since there were no takers i went ahead and did it.

The repo at  http://code.google.com/p/netmap/ has been
updated with a newer version of netmap, and the extra/
directory contains a netmap backend for libpcap
https://code.google.com/p/netmap/source/browse/extra/libpcap-netmap.diff

which works against a recent (Jan11) libpcap version from github

https://github.com/the-tcpdump-group/libpcap/commits/master

commit fd04c4ff9f9a6b50fccec7afb18af64433730a2b
Author: Guy Harris <guy () alum mit edu>
Date:   Sat Jan 11 20:38:02 2014 -0800

For some quick testing (linux; FreeBSD is similar):

- download the netmap code from code.google.com/p/netmap/

- compile just the netmap module and the examples

   (cd LINUX; make NODRIVERS=1 )
   (cd examples; make)

- install the module
   (cd LINUX; sudo insmod ./netmap_lin.ko)
  (either change access privs to /dev/netmap, or
  run netmap clients with root privs)

- fetch the pcap code

- patch with the netmap support files

   cd pcap-base-code;
   patch -p1 < /wherever/is/the/netmap-base-dir/extra/libpcap-netmap-diff )

- make sure the netmap headers are accessible and rebuild libpcap

   export CFLAGS=-I/wherever/is/the/netmap-base-dir/sys
   ./configure
   make

- create a link so ld will find it
   ln -s libpcap.so.1.6.0-PRE-GIT libpcap.so.0.8

- and now you can run tcpdump using the current library
  (depending on the OS you may need to tell apparmor to
  allow the library override for tcpdump, or make a non suid
  copy of tcpdump)

   LD_LIBRARY_PATH=. tcpdump -ni valexx:yy

   while in another window you run a netmap traffic generator

   /wherever/is/the/netmap-base-dir/pkt-gen -i valexx:zz -f tx

You can access an ordinary interface in emulated netmap mode by
prefixing the name with netmap: , but BEWARE:
*** in this mode the interface is only used for capture,
*** and goes back to regular mode when you exit the tcpdump

   LD_LIBRARY_PATH=. tcpdump -ni netmap:eth0

cheers
luigi


On Wed, Dec 4, 2013 at 7:27 PM, Luigi Rizzo <rizzo () iet unipi it> wrote:

> Hi,
> i have recently made an update to the netmap I/O framework
>
>         http://info.iet.unipi.it/~luigi/netmap/
>
> that should make it easier to add netmap support to libpcap.
> So I was wondering if there is any interest to implement this
> and how we can go for it.
>
> In short (see the webpage for details) netmap is a kernel module
> (native on FreeBSD, external in Linux) that supports extremely high
> tx/rx packet rates (15-20Mpps per core, at least for the raw I/O;
> of course any processing will reduce your actual packet rate).
> You can find the most recent sources in the git repository at
>
>         http://code.google.com/p/netmap/
>
> In the past I have implemented a subset of the pcap library that
> lets programs run on top of netmap by just pointing LD_LIBRARY_PATH
> to the netmap-based library.  This has some limitations though,
> and I'd rather see native netmap support in libpcap so we can
> e.g. reuse filters etc.
>
>
> A basic implementation of the equivalent of pcap_open_live(),
> pcap_close(), pcap_dispatch() and pcap_next() is in the <net/netmap_user.h>
> header file (it is so small and simple that there is really no need
> for a user library). pcap_inject() is similarly simple.
>
> Of course they should be integrated with libpcap and support
> the full set of methods, so i think to figure out the following:
>
> PORTING ISSUES
>
> + interface naming
>   netmap provides an alternate method to access standard network
>   interfaces, so the technique I am currently using in applications
>   is to use the interface name to discriminate between standard
>   (bpf, PF_PACKET or other) and netmap mode.
>   This way applications do not need changes, and commandline
>   arguments can be used to select the operating mode.
>
>   "netmap:*" refers to interfaces in netmap mode, "valeXX:YY" refers
>   to ports of VALE virtual switches (basically dynamically created
>   ethernet bridges; VALE is part of the netmap module), and other
>   names would just fall back to the regular pcap methods.
>
>   Does this make sense ?
>
> + template for source
>   I suppose the way to go is to pick the simplest pcap-*.c backend
>   and use it as a reference for the implementation -- so which one
>   should I use ?
>
> + receive side
>   netmap natively uses shared memory, so pcap_dispatch() and
>   pcap_next() are trivial to implement and very cheap.
>
> + transmit side
>   pcap_inject() can be implemented easily by copying user data
>   into the (preallocated) buffers supplied by netmap.
>
> + zerocopy
>   As mentioned the receive side is alrea
> dy
> zerocopy,
>   while for the transmit side i don't know if there is a pcap
>   method that support a transmit callback -- i.e. an equivalent
>   of pcap_dispatch() where pcap supplies the buffer and the
>   callback puts in its data.
>   Also one thing to remember is the address(es) returned by
>   pcap_next() are only good until the next invocation of
>   select()/poll() . Again I have no idea if the pcap API
>   gives some control to the user for that
>
> + multiqueue
>   netmap supports multiqueue NICs both on tx and rx.
>   There are two operating modes: 1) one file descriptor
>   binds to all queues; 2) one file descriptor per queue.
>   This is selected at open time.
>   Supporting this feature could be as simple as adding
>   a suffix to the interface name.
>
>
> Comments anyone ?
>
> cheers
> luigi


-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo () iet unipi it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2211611               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers