Re: massping migration and you

[David Fifield <david () bamsoftware com>]

On Fri, Aug 31, 2007 at 10:31:45PM +0000, Brandon Enright wrote:
> Okay, a correction, comments, and test results below.  This should be a lot
> more to your liking.
>
> Nmap done: 186368 IP addresses (15804 hosts up) scanned in 258.023 seconds
>                Raw packets sent: 1398404 (55.937MB) | Rcvd: 51659 (2.426MB)
>
> real    4m18.043s
> user    1m45.858s
> sys     0m35.164s
>
> With the default parallelism used with -T5 the new ultrascan() code is much
> slower and more conservative that the massping() code was.  It can be made
> to be MUCH faster though (see above!).  I wasn't able to make massping() go
> any faster so ultrascan() really seems to be showing it's power and
> flexibility.

All right. That's good news. I'd like to see ultra_scan work more like
that without having to give any special options like --min-parallelism.
I have a feeling that the biggest thing making your previous tests slow
was the drops you were getting. What --min-parallelism does is reduce
the potential impact of drops by keeping the congestion window above a
certain size.

It may be time to rethink how the group congestion window behaves,
because it's such a limiting factor for scans like this. Especially for
ping scan, a drop from one target shouldn't have such a drastic effect
on all the others as halving the congestion window. The window for that
one target drops to 1 anyway. Perhaps the group congestion window should
drop by only 20% (say) rather than 50% one a drop, and recover much more
quickly in congestion avoidance mode.

> > Can you do one of your scans like
> >
> >     nmap -n -d2 --packet-trace | grep -B 3 DROPPED
> >
> > I want to know what the nature of the drops is. There are conditions
> > under which a good response can be counted as a drop.
>
> Okay.  For all the following scans, I used -n -d2 --packet-trace (and other
> options).  I did get some DROPPED where HOST_UP -> HOST_UP, and
> interestingly, the only the medium parallelism scans produced a few.
>
> Here is the template for the scans I ran:
>
> $ time sudo ./nmap -sP -P A135,139,445,3389 -n -vv -d2 -T5 --packet-trace
> - --min-parallelism X --max-parallelism Y --excludefile <a file> a.b.0.0/16
> c.d.0.0/16 e.f.0.0/16
>
> First the DROPPED HOST_UP -> HOST_UP results:
>
> minX: 64, maxY: 64
> 2
>
> minX: 128, maxY: 128
> 6
>
> minX: 256, maxY: 256
> 8
>
> minX: 512, maxY: 512
> 6
>
> minX: 768, maxY: 768
> 1

Okay. That these are the minority of detected drops agrees with the
testing I did. That part of the code should still be fixed.

> Now onto the time per scan and host up results.  Don't read too much into
> the number of hosts detected as up for each scan.  Not all of these scans
> were done in the order presented here.
>
> ...

Thanks for your detailed tests! They have been very helpful.

> To get and idea of the trade-off between speed and accuracy, I started two
> additional scans at the same time.  One with a parallelism of 256 and the
> other with 2048.  Here are those results:
>
> 256:
> Nmap done: 186368 IP addresses (15505 hosts up) scanned in 653.847 seconds
>             Raw packets sent: 1395793 (55.832MB) | Rcvd: 245905 (11.801MB)
>
> 2048:
> Nmap done: 186368 IP addresses (15279 hosts up) scanned in 311.655 seconds
>             Raw packets sent: 1402726 (56.109MB) | Rcvd: 154293 (7.459MB)

That's quite a big difference in both time and accuracy. Then again, 653
seconds is faster than any of your massping scans, right? The problem
with --min-parallelism is that it's a hard lower bound--the scan can't
go slower even if it really needs to. This puts the human in charge of
picking parameters instead of the program doing it automatically based
on network conditions. I take this as more evidence that the group
congestion control variables should behave differently in the face of
drops.

> I think the moral of the story is that if you want to reach (or far
> surpass) the speed of the old massping() code, --min-parallelism is your
> friend.  Of course this comes at an accuracy trade-off.  Also, even with
> - -T5, the new ultrascan() code is dismally slow at host-discovery without a
> - --min-parallelism of at least 32-64.

People expect -T5 to make just such an accuracy-speed trade-off, so
maybe -T5 should increase the minimum parallelism.

David

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org