Re: massping-migration and other dev testing results

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 14 Sep 2007 15:37:04 -0600 plus or minus some time David Fifield
<david () bamsoftware com> wrote:

I wrote:
> Okay, did that.  To recap, my 'b' scan is '-sP -P A135,139,445,3389'
> across 180k hosts.
>
> I did this scan with MPM r5829 twice, sequentially, with no other network
> traffic or CPU load on the box.  Once with T3 and once with T5.
...snip...
You wrote:
> Weird, there should be many more host groups than that. At least
> 180000 / 4096 ~= 44, instead of only 6. Did you run these scans with
> enlarged ping groups?

My SVN tree got corrupted somehow and wasn't handling (updating) locally
modified files.  Here are the correct results now:

david_mpm_r5829bT3.nmap:
# Nmap done at Sat Sep 15 04:38:45 2007 -- 186368 IP addresses (14123 hosts
up) scanned in 4977.963 seconds

david_mpm_r5829bT5.nmap:
# Nmap done at Sat Sep 15 03:15:47 2007 -- 186368 IP addresses (13841 hosts
up) scanned in 2571.266 seconds

The graphs have been re-generated.

$ egrep 'Ultrascan DROPPED' david_mpm_r5829bT3.nmap | wc -l
376

$ egrep 'Ultrascan DROPPED' david_mpm_r5829bT5.nmap | wc -l
490

And for the long pcap results:

$ egrep -i 'pcap stats' david_mpm_r5829bT3.nmap
pcap stats: 131 packets received by filter, 0 dropped by kernel.
pcap stats: 18 packets received by filter, 0 dropped by kernel.
pcap stats: 44 packets received by filter, 0 dropped by kernel.
pcap stats: 872 packets received by filter, 0 dropped by kernel.
pcap stats: 1376 packets received by filter, 0 dropped by kernel.
pcap stats: 938 packets received by filter, 0 dropped by kernel.
pcap stats: 1356 packets received by filter, 0 dropped by kernel.
pcap stats: 8624 packets received by filter, 0 dropped by kernel.
pcap stats: 912 packets received by filter, 0 dropped by kernel.
pcap stats: 1193 packets received by filter, 0 dropped by kernel.
pcap stats: 1506 packets received by filter, 0 dropped by kernel.
pcap stats: 823 packets received by filter, 0 dropped by kernel.
pcap stats: 1426 packets received by filter, 0 dropped by kernel.
pcap stats: 999 packets received by filter, 0 dropped by kernel.
pcap stats: 937 packets received by filter, 0 dropped by kernel.
pcap stats: 1219 packets received by filter, 0 dropped by kernel.
pcap stats: 602 packets received by filter, 0 dropped by kernel.
pcap stats: 534 packets received by filter, 0 dropped by kernel.
pcap stats: 561 packets received by filter, 0 dropped by kernel.
pcap stats: 425 packets received by filter, 0 dropped by kernel.
pcap stats: 107 packets received by filter, 0 dropped by kernel.
pcap stats: 53 packets received by filter, 0 dropped by kernel.
pcap stats: 403 packets received by filter, 0 dropped by kernel.
pcap stats: 195 packets received by filter, 0 dropped by kernel.
pcap stats: 947 packets received by filter, 0 dropped by kernel.
pcap stats: 427 packets received by filter, 0 dropped by kernel.
pcap stats: 180 packets received by filter, 0 dropped by kernel.
pcap stats: 451 packets received by filter, 0 dropped by kernel.
pcap stats: 219 packets received by filter, 0 dropped by kernel.
pcap stats: 517 packets received by filter, 0 dropped by kernel.
pcap stats: 173 packets received by filter, 0 dropped by kernel.
pcap stats: 447 packets received by filter, 0 dropped by kernel.
pcap stats: 246 packets received by filter, 0 dropped by kernel.
pcap stats: 104 packets received by filter, 0 dropped by kernel.
pcap stats: 75 packets received by filter, 0 dropped by kernel.
pcap stats: 133 packets received by filter, 0 dropped by kernel.
pcap stats: 1329 packets received by filter, 0 dropped by kernel.
pcap stats: 578 packets received by filter, 0 dropped by kernel.
pcap stats: 199 packets received by filter, 0 dropped by kernel.
pcap stats: 8790 packets received by filter, 0 dropped by kernel.
pcap stats: 389 packets received by filter, 0 dropped by kernel.
pcap stats: 2018 packets received by filter, 0 dropped by kernel.
pcap stats: 503 packets received by filter, 0 dropped by kernel.
pcap stats: 281 packets received by filter, 0 dropped by kernel.
pcap stats: 65 packets received by filter, 0 dropped by kernel.
pcap stats: 79 packets received by filter, 0 dropped by kernel.
pcap stats: 185 packets received by filter, 0 dropped by kernel.
pcap stats: 352 packets received by filter, 0 dropped by kernel.
pcap stats: 96 packets received by filter, 0 dropped by kernel.


$ egrep -i 'pcap stats' david_mpm_r5829bT5.nmap
pcap stats: 138 packets received by filter, 0 dropped by kernel.
pcap stats: 18 packets received by filter, 0 dropped by kernel.
pcap stats: 43 packets received by filter, 0 dropped by kernel.
pcap stats: 1086 packets received by filter, 0 dropped by kernel.
pcap stats: 1563 packets received by filter, 0 dropped by kernel.
pcap stats: 930 packets received by filter, 0 dropped by kernel.
pcap stats: 1677 packets received by filter, 0 dropped by kernel.
pcap stats: 762 packets received by filter, 0 dropped by kernel.
pcap stats: 920 packets received by filter, 0 dropped by kernel.
pcap stats: 842 packets received by filter, 0 dropped by kernel.
pcap stats: 1206 packets received by filter, 0 dropped by kernel.
pcap stats: 825 packets received by filter, 0 dropped by kernel.
pcap stats: 1113 packets received by filter, 0 dropped by kernel.
pcap stats: 795 packets received by filter, 0 dropped by kernel.
pcap stats: 695 packets received by filter, 0 dropped by kernel.
pcap stats: 832 packets received by filter, 0 dropped by kernel.
pcap stats: 512 packets received by filter, 0 dropped by kernel.
pcap stats: 239 packets received by filter, 0 dropped by kernel.
pcap stats: 133 packets received by filter, 0 dropped by kernel.
pcap stats: 87 packets received by filter, 0 dropped by kernel.
pcap stats: 80 packets received by filter, 0 dropped by kernel.
pcap stats: 99 packets received by filter, 0 dropped by kernel.
pcap stats: 63 packets received by filter, 0 dropped by kernel.
pcap stats: 78 packets received by filter, 0 dropped by kernel.
pcap stats: 1273 packets received by filter, 0 dropped by kernel.
pcap stats: 430 packets received by filter, 0 dropped by kernel.
pcap stats: 131 packets received by filter, 0 dropped by kernel.
pcap stats: 445 packets received by filter, 0 dropped by kernel.
pcap stats: 195 packets received by filter, 0 dropped by kernel.
pcap stats: 153 packets received by filter, 0 dropped by kernel.
pcap stats: 160 packets received by filter, 0 dropped by kernel.
pcap stats: 406 packets received by filter, 0 dropped by kernel.
pcap stats: 127 packets received by filter, 0 dropped by kernel.
pcap stats: 112 packets received by filter, 0 dropped by kernel.
pcap stats: 110 packets received by filter, 0 dropped by kernel.
pcap stats: 152 packets received by filter, 0 dropped by kernel.
pcap stats: 2572 packets received by filter, 642 dropped by kernel.
pcap stats: 712 packets received by filter, 0 dropped by kernel.
pcap stats: 148 packets received by filter, 0 dropped by kernel.
pcap stats: 149 packets received by filter, 0 dropped by kernel.
pcap stats: 141 packets received by filter, 0 dropped by kernel.
pcap stats: 149 packets received by filter, 0 dropped by kernel.
pcap stats: 83 packets received by filter, 0 dropped by kernel.
pcap stats: 32 packets received by filter, 0 dropped by kernel.
pcap stats: 42 packets received by filter, 0 dropped by kernel.
pcap stats: 58 packets received by filter, 0 dropped by kernel.
pcap stats: 87 packets received by filter, 0 dropped by kernel.
pcap stats: 52 packets received by filter, 0 dropped by kernel.
pcap stats: 59 packets received by filter, 0 dropped by kernel.


Other than the one drop spike, everything went fine.  Is there any way to
figure out why the kernel would drop received packets?  My best guess would
be that there is a pretty short buffer for incoming packets and if too many
probes are sent at once, when responses come back, if Nmap takes to long to
read them from the buffer, packets will be dropped.  That would help
explain why with the 64k PING_GROUP_SZ the kernel was dropping like crazy
- -- Nmap was spending too much time sending and the latency to getting to the
buffer read was too high.  Or maybe it's something completely different.
Thoughts?

Brandon



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFG63cJqaGPzAsl94IRArQDAJ4+r9hCH6EIuu0qsj67GfFLFgRzwgCfZ7v6
xopitH5fSD2ek67lgq7NxP4=
=1rWA
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org