Nmap Development mailing list archives
Re: massping-migration and other dev testing results
From: David Fifield <david () bamsoftware com>
Date: Tue, 11 Sep 2007 12:36:05 -0600
On Tue, Sep 11, 2007 at 07:55:28AM +0000, Brandon Enright wrote:
Okay, I have testing results finished. To recap from the last few weeks, David wanted to compare the speed of host discovery with the current release (4.20) with the new migrated code in the svn nmap branch and some experimental congestion control changes in the massping-migration branch. I triple checked that I was using the right options with the right version of Nmap. I'm presenting the scan in the order that they were done. Since these results represent a significant amount of time scanning, a decrease of about 2000 hosts discovered is to be expected expected from start to finish. For each group, I did two scans. A scan 'a' and a scan 'b'. Scan 'a' had a min/max parallelism of 1024. Scan 'b' just had -T5 specified. massping-migration branch results first: r5815 has all the tweaks. massping/r5815a: # Nmap done at Tue Sep 11 00:13:17 2007 -- 186368 IP addresses (10582 hosts up) scanned in 995.604 seconds massping/r5815b: # Nmap done at Tue Sep 11 00:52:54 2007 -- 186368 IP addresses (12586 hosts up) scanned in 3371.708 seconds r5786 does not have the cap of 50 on the cwnd scaling factor massping/r5786a: # Nmap done at Tue Sep 11 01:11:46 2007 -- 186368 IP addresses (10214 hosts up) scanned in 1000.378 seconds massping/r5786b: # Nmap done at Tue Sep 11 01:48:37 2007 -- 186368 IP addresses (12164 hosts up) scanned in 3210.024 seconds
It makes sense that these should be about the same. The version without the cap is a little faster, but it doesn't make much difference.
r5780 does not scale with the inverse of the packet receipt ratio massping/r5780a: # Nmap done at Tue Sep 11 02:20:55 2007 -- 186368 IP addresses (9884 hosts up) scanned in 957.285 seconds massping/r5780b: # Nmap done at Tue Sep 11 04:27:53 2007 -- 186368 IP addresses (13667 hosts up) scanned in 8573.451 seconds Now for the latest nmap SVN branch: nmap/r5817a: # Nmap done at Tue Sep 11 05:25:43 2007 -- 186368 IP addresses (11211 hosts up) scanned in 274.352 seconds nmap/r5817b: # Nmap done at Tue Sep 11 07:16:00 2007 -- 186368 IP addresses (14122 hosts up) scanned in 5556.831 seconds
This is unexpected. The only difference between current /nmap and r5780 from massping-migration with regard to scanning is that r5780 doesn't adjust congestion parameters for ICMP destination unreachables. You can verify this yourself with svn diff svn://svn.insecure.org/nmap@5817 svn://svn.insecure.org/nmap-exp/david/nmap-massping-migration@5780 That could account for the difference between 8573 and 5557 seconds, but not for the difference between 957 and 274 seconds. The effect of ignoring or not ignoring destination unreachables is completely negated when you set the min and max parallelism to be equal, a fact which your david_mpm_r5780a.svg supports. Actually another thought just occurred to me. The nmap-massping-migration branch also has debugging and profiling flags turned on. I wonder if that could have such a large effect? I've noticed that these flags make Nmap use a lot more CPU on my system, but I haven't measured them to actually slow down any scans. I'll remove those flags in preparation for the next few tests I'd like you to do (see below).
Now for the latest release of nmap (4.20) results. The a and b scans are the same as the previous a/b. The c replaced --min-parallelism 1024 with - --min-hostgroup 1024 and the d uses a --min-hostgroup of 2048: 4.20a: # Nmap run completed at Tue Sep 11 05:40:03 2007 -- 186368 IP addresses (13257 hosts up) scanned in 2561.333 seconds 4.20b: # Nmap run completed at Tue Sep 11 05:52:08 2007 -- 186368 IP addresses (13235 hosts up) scanned in 3285.450 seconds 4.20c: # Nmap run completed at Tue Sep 11 06:40:57 2007 -- 186368 IP addresses (13213 hosts up) scanned in 3436.940 seconds 4.20d: # Nmap run completed at Tue Sep 11 07:44:46 2007 -- 186368 IP addresses (13168 hosts up) scanned in 3268.226 seconds This is a lot of data recorded over almost 12 hours of time. I'm more than willing to run any two of these scans side-by-side to get a better idea for their speed and accuracy when the network is the same for both scans. Or if there is another SVN revision or scan args I should try, let me know.
I have two requests: First, please generate graphs from your nmap/r5817 tests. The /nmap branch prints enough information to make the graphs. Second, please run side-by-side the latest /nmap (the same as your test above) and the latest nmap-massping-migration (r5824, the same as above the exception of the removal of debugging flags). Run your 'a' tests concurrently, then your 'b' tests concurrently.
- From these results, I'd say that the SVN nmap branch is both the fastest, more accurate, and most adjustable. Two things strike me as odd in these tests. The first is that the congestion control ideas implemented in massping-migration sound excellent but perform poorly. The second is that identical scans with nmap 4.20 took twice as long in this test. I'm going to experiment more to try to reproduce the release version scanning our network in about 23 minutes on average.
I wouldn't say it performs poorly when it beats nmap trunk with unspecified parallelism (3372 vs. 5557 seconds), assuming the difference of 1536 hosts is due to the difference of six hours between the scans. I hope to isolate whether the debugging flags were slowing down the tests with a high parallelism, although I think that use case is secondary to having fast scans without specifying the parallelism. It would be pretty sweet to have the best of both worlds: 180,000 hosts scanned in an hour with high accuracy, or 180,000 hosts scanned in under five minutes (!) with a 20% loss in accuracy. That's a lot of flexibility compared with massping, and I think it's attainable. It is curious that the 4.20 scans have such a large difference from before.
To aid in figuring out what is going on, I've generated graphs for each of the massping-migration tests. They are at http://noh.ucsd.edu/~bmenrigh/nmap/
It looks like you do have a bunch of blocks with no hosts up in them. The congestion window is stuck at 10 for the length of those blocks. --randomize-hosts might be good option for you, as it both increases the group size and might mix some live hosts into those long empty stretches. But please leave things as they are for your current tests.
I have a few more tests I want to run but in the mean time, if anyone has any questions or requests, let me know.
Thanks so much for your help so far. Has anyone else given this a try? David Fifield _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- massping-migration and other dev testing results Brandon Enright (Sep 11)
- Re: massping-migration and other dev testing results David Fifield (Sep 11)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 11)
- Re: massping-migration and other dev testing results David Fifield (Sep 11)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 11)
- Re: massping-migration and other dev testing results David Fifield (Sep 13)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 13)
- Re: massping-migration and other dev testing results David Fifield (Sep 14)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 14)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 14)
- Re: massping-migration and other dev testing results David Fifield (Sep 17)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 11)
- Re: massping-migration and other dev testing results David Fifield (Sep 11)
- Re: massping-migration and other dev testing results Brandon Enright (Sep 11)