pcap-int.h, Packet functions on Windows

[David Fifield <david () bamsoftware com>]

There's this TODO item:

o Investigate why and whether we need mswin32/pcap-include/pcap-int.h.
  This file is not included in the official WinPcap 4.1.1 developers'
  pack (http://www.winpcap.org/install/bin/WpdPack_4_1_1.zip).
  Presumably it covers internal functions and structures which we aren't
  really supposed to access it.  If we can get rid of it, that would be
  great.

We only use this pcap-int.h in a couple of places, and only on Windows.
It's used in a couple of places so that we can break the abstraction of
struct pcap and access the pcap.adapter member, in order to set the read
timeout with PacketSetReadTimeout. This function is part of the Packet
interface to WinPcap, which is discouraged in favor of the normal
libpcap interface.

PacketSetReadTimeout is only called in two places that are essentially
different:
  nsock/src/nsock_pcap.c
  tcpip.cc and nping/common_modified.cc (these are the same)

The easy case first: I think the call to PacketSetReadTimeout in
nsock_pcap.c can simply be removed, as it's in the same function that
calls pcap_open_live, and pcap_open_live already calls
PacketSetReadTimeout. I'm going to remove it once I double-check this.

Why is PacketSetReadTimeout being called in tcpip.cc? It's because
pcap_select doesn't work on WinPcap handles. On some platforms, we use
pcap_select to enforce a timeout on reads. This is the code in tcpip.cc:
    if (pcap_select(pd, to_usec) == 0)
      timedout = 1;
    else
      p = (char *) pcap_next(pd, &head);
On Linux, pcap_select enforces the timeout and pcap_next returns
immediately, while on Windows it's the opposite: pcap_select returns -1
immediately and pcap_next enforces the timeout set by
PacketSetReadTimeout.

We do this because we potentially want to set a different timeout on
every read. The libpcap interface only allows setting a timeout when the
device is opened.

The thing is, other platforms, including OS X, also do not have
selectable pcap file handles. We handle these by setting a short (2 ms)
timeout when the device is opened, and calling pcap_next in a loop until
the timeout is up. Essentially, we're polling instead of waiting to be
notified. This same technique works on Windows if we just remove the
calls to PacketSetReadTimeout. The question is, does it hurt performance
or accuracy? These are tests I did today.

readip_pcap: Number of times readip_pcap is called.
pcap_next: Number of times pcap_next is called by readip_pcap.
avg: pcap_next / readip_pcap
real, user, sys: output of "time" command.

This is a scan of an unassigned IP, so all ports are filtered. Removing
PacketSetReadTimeout explodes the number of calls to pcap_next, because
we're polling every 2 ms instead of waiting for a nice long timeout.
However, real time and CPU time are virtually unchanged.

# nmap.exe -n -Pn 192.168.1.10 -F
With PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
         43         43  1.00 25.203 0.015 0.000
         43         43  1.00 24.953 0.015 0.015
         43         43  1.00 24.937 0.031 0.000
         43         43  1.00 24.969 0.015 0.015
         43         43  1.00 24.953 0.031 0.000
Without PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
         43       1366 31.77 25.719 0.015 0.000
         43       1366 31.77 24.953 0.015 0.015
         43       1366 31.77 24.953 0.015 0.015
         43       1366 31.77 24.953 0.030 0.000
         43       1365 31.74 24.938 0.015 0.015

In this scan of scanme, we get some feedback from the network, so
timeouts are shorter. That means even though we are polling, we don't
call pcap_next as much as with the default timeouts. Removing
PacketSetReadTimeout seems to make the scan take longer in 3 out of 5
cases. My hypothesis is that the polling method is less accurate,
causing us to miss some incoming packets, leading to detected drops
which increase the timeouts. Notice how the pcap_next column is much
higher in the scans that took longer. Higher readip_pcap counts also
indicate that we were starting to use up retries.

# nmap.exe -n -Pn scanme.nmap.org
With PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
        135        135  1.00 11.515 0.015 0.015
        208        208  1.00 14.891 0.015 0.015
        135        135  1.00 11.484 0.015 0.015
        130        130  1.00 11.297 0.015 0.015
        153        153  1.00 14.531 0.015 0.015
Without PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
        393       1877  4.78 32.953 0.015 0.015
        130        344  2.65  9.203 0.031 0.000
        367       1566  4.27 27.812 0.015 0.016
        225        962  4.28 18.579 0.015 0.015
        158        514  3.25 11.671 0.015 0.015

Here's a scan of a LAN host with all ports responsive (either open or
closed). Here the timeout drops below 2 ms quickly, so removing
PacketSetReadTimeout has no effect.

# nmap.exe -n -Pn 192.168.0.1
With PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
       1009       1009  1.00  5.078 0.015 0.015
       1004       1004  1.00  4.547 0.031 0.000
       1007       1007  1.00  4.563 0.031 0.000
       1007       1007  1.00  4.547 0.031 0.000
       1003       1003  1.00  4.562 0.015 0.015
Without PacketSetReadTimeout
readip_pcap  pcap_next   avg   real  user   sys
       1003       1003  1.00  4.766 0.015 0.000
       1005       1005  1.00  4.531 0.015 0.015
       1005       1005  1.00  4.547 0.015 0.015
       1004       1004  1.00  4.547 0.015 0.015
       1010       1010  1.00  4.531 0.015 0.015

These results are encouraging in that removing PacketSetReadTimeout
seems to have no direct effect on real or CPU time. But it seems it is a
less accurate packet capturing mechanism that can lead to drops and
slower scans. So I suggest that we leave this code in tcpip.cc until
another hypothesis as to the slowdown is offered.

The patch I used to test is attached.

David Fifield

Attachment: PacketSetReadTimeout-counts.diff
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/