Re: massping-migration and other dev testing results

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'll cut down the bits of my email here for a followup on adjusting
PING_GROUP_SZ.

On Wed, 12 Sep 2007 02:07:19 +0000 plus or minus some time Brandon Enright
<bmenrigh () ucsd edu> wrote:
> I was under the impression that --randomize-hosts only randomizes host
> within a ping group.  The fact that it also increases the size of the
> ping group is *huge*.  The man doesn't make this all that clear.  I went
> ahead and looked at the code for this and I have a couple of thoughts.
>
> First, PING_GROUP_SZ is set to 4096.  When you use --randomize-hosts
> 'o.ping_group_sz = PING_GROUP_SZ * 4;' is run.  The man says the group can
> grow to up to 8096.  There doesn't appear to be any special cap so the
> group size would actually be 16384.
>
> Second, it really surprises me that an important value like this isn't
> adjustable.  I thought --min-hostgroup set the ping group size but after
> looking at the code, this doesn't appear to be the case.  I suppose most
> people aren't scanning 10k+ hosts so it doesn't matter much.  For those
> that do though, it really matters.
>
> Since this value is already so large, using the value from --min-hostgroup
> is probably not a good idea.  Perhaps another option like
> --min-ping-group.
>
> I have preliminary results that show larger ping_groups (using either
> - --randomize-hosts, or recompiling) to really help.  Some of the scans
> are still going though so I'll have to send a follow up email
> illustrating this when I get home.
>
> I'm going to follow up with tweaked PING_GROUP_SZ results but here is a
> preview. I ran david_mpm_r5824b.nmap (took 2640 seconds) with
> - --randomize-hosts and shaved off 600 seconds:
>
> david_mpm_r5824c.nmap:
> # Nmap done at Wed Sep 12 01:27:32 2007 -- 186368 IP addresses (13327
> hosts up) scanned in 2019.837 seconds
>
> This scan did find 2k fewer hosts, but since they were done around 5pm
> local time some of this drop-off is hosts being turned off.

Okay, so I have results for adjusting the ping group sizes.

Here are The two base scans with no adjustment.  These used -T5 only:

david_mpm_r5824b.nmap:
# Nmap done at Wed Sep 12 00:17:31 2007 -- 186368 IP addresses (15628 hosts
up) scanned in 2640.259 seconds

david_nmap_r5824b.nmap:
# Nmap done at Wed Sep 12 00:49:08 2007 -- 186368 IP addresses (15901 hosts
up) scanned in 4536.876 seconds


Here are those scans repeated with --randomize-hosts to increase the ping
group from 4096 to 16384:

david_mpm_r5824c.nmap:
# Nmap done at Wed Sep 12 01:27:32 2007 -- 186368 IP addresses (13327 hosts
up) scanned in 2019.837 seconds

david_nmap_r5824c.nmap:
# Nmap done at Wed Sep 12 02:01:47 2007 -- 186368 IP addresses (14696 hosts
up) scanned in 4073.890 seconds


Now I modified PING_GROUP_SZ to be 65536 and didn't use --randomize-hosts:

# Nmap done at Wed Sep 12 02:53:11 2007 -- 186368 IP addresses (6293 hosts
up) scanned in 2654.588 seconds

# Nmap done at Wed Sep 12 02:58:49 2007 -- 186368 IP addresses (4801 hosts
up) scanned in 2992.511 seconds

Ouch, that really hurt accuracy and actually slowed down the MPM branch.
Clearly there is going to be some sweet spot and 2**16 is overkill.
Maybe 4096 is good.  Maybe 8192 is better?  I'll have to test this.

Brandon



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFG53z2qaGPzAsl94IRAua5AKCZUKI3DqTuhndufNBgEMVPCrHSswCeLG3o
q2Lh3PNTxFtKxFthSbo/9Us=
=4Sxh
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org