Re: massping-migration and other dev testing results

[David Fifield <david () bamsoftware com>]

On Fri, Sep 14, 2007 at 04:41:26AM +0000, Brandon Enright wrote:
> On Thu, 13 Sep 2007 11:53:34 -0600 plus or minus some time David Fifield
> <david () bamsoftware com> wrote:
> > > I agree that lose is occurring somewhere, I just don't think it is the
> > > fault of the network.  I've seen other tools that use libpcap report
> > > dropped packets once in a while.  Is it possible that Nmap either isn't
> > > getting the packets out and they are being dropped by libpcap or that
> > > the responses are getting dropped on the way in?
> >
> > To investigate this, I added a function to the massping migration branch
> > that prints the number of dropped packets reported by libpcap. With -d2,
> > it's called once per invocation of ultra_scan, so roughly once per 4096
> > hosts during host discovery.
> >
> > Please run your mpm 'b' scan again with -T5 and see if there are any
> > drops (the stats lines start with "pcap stats:"). Then run it with -T3
> > and see if more hosts are detected in the (presumably) longer time the
> > scan takes.
>
> Okay, did that.  To recap, my 'b' scan is '-sP -P A135,139,445,3389' across
> 180k hosts.
>
> I did this scan with MPM r5829 twice, sequentially, with no other network
> traffic or CPU load on the box.  Once with T3 and once with T5.
>
> david_mpm_r5829bT3.nmap:
> # Nmap done at Fri Sep 14 04:14:31 2007 -- 186368 IP addresses (12502 hosts
> up) scanned in 4032.982 seconds
>
> david_mpm_r5829bT5.nmap:
> # Nmap done at Fri Sep 14 03:07:18 2007 -- 186368 IP addresses (7773 hosts
> up) scanned in 2519.749 seconds
>
> Pretty scary how many more hosts -T3 found.  I don't really understand this
> considering the packet loss over the actual network should be 0 and the
> latency less than 5 ms.  Are hosts really that slow to respond?

Wow, that's way different from your previous -T5 test, which should have
been the same:

> david_mpm_r5824b.nmap:
> # Nmap done at Wed Sep 12 00:17:31 2007 -- 186368 IP addresses (15628 hosts
> up) scanned in 2640.259 seconds

>  $ egrep -i 'pcap stats' david_mpm_r5829bT3.nmap
> pcap stats: 115 packets received by filter, 0 dropped by kernel.
> pcap stats: 18 packets received by filter, 0 dropped by kernel.
> pcap stats: 43 packets received by filter, 0 dropped by kernel.
> pcap stats: 53553 packets received by filter, 5614 dropped by kernel.
> pcap stats: 13849 packets received by filter, 769 dropped by kernel.
> pcap stats: 7488 packets received by filter, 272 dropped by kernel.
>
> $ egrep -i 'pcap stats' david_mpm_r5829bT5.nmap
> pcap stats: 139 packets received by filter, 0 dropped by kernel.
> pcap stats: 18 packets received by filter, 0 dropped by kernel.
> pcap stats: 43 packets received by filter, 0 dropped by kernel.
> pcap stats: 39723 packets received by filter, 223 dropped by kernel.
> pcap stats: 9289 packets received by filter, 46 dropped by kernel.
> pcap stats: 7515 packets received by filter, 699 dropped by kernel.

Weird, there should be many more host groups than that. At least
180000 / 4096 ~= 44, instead of only 6. Did you run these scans with
enlarged ping groups?

> It is interesting that in only two of the groups in -T5 were fewer packets
> received than in -T3.  I also find it concerning that the kernel dropped
> more packets in -T3; or that the kernel is dropping packets at all.

The results are surprising, at any rate. Does the slower pace of -T3 let
packets sit in the pcap buffer for too long before the get handled,
maybe?

Despite these weirdnesses, it looks like you're able to get satisfactory
results.

> I've generating graphs for these scans, available at
> htpp://noh.ucsd.edu/~bmenrigh/nmap/

How weird that david_mpm_r5829bT3.svg blasts off to 300 for a stretch.
It looks like it hit a good timing ping host with no other hosts
responding for a while. Responses from timing pings count more, which
accounts for the increased slope.

David

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org