Re: New development in host discovery: response rate scaled congestion control

[David Fifield <david () bamsoftware com>]

> > You may have been following the development of ultra_scan-based host
> > discovery. In the last few weeks, we've been working to improve its
> > performance. Some testing reports have shown slow performance when many
> > packet drops are detected. I recently hit on an idea that could greatly
> > improve the situation in these cases.
>
> I've been poking at the SVN code for a few hours now and I've gotten some
> odd results.
>
> Here is an example:
>
> nmap -sP -P A135,139,445,3389 -n -v -v -d3 -T5 --min-parallelism X
> - --max-parallelism Y --excludefile <a file> -oA <file>
> a.b.0.0/16 c.d.0.0/16 e.f.0.0/16 --packet-trace
>
> I started three copies of this scan with 64-64, 64-2048, and 2048-2048 for
> the parallelism range.  The results all finished within about 1 second of
> each other.  When I look at the output, it is apparent that they all
> crashed within about 1 second of each other.  The really odd thing is this
> happened 33 minutes in for each scan.  Even without a crash, 33 minutes is
> a lot longer than these parallelism windows were taking before.

I'll check this out. That they all crashed at the same time is curious.

Can you try running it with -d2 instead of -d3? -d3 is extremely
verbose. It prints out timing stats for every single host in each block
of 4096 many times a second. That alone might be slowing the scan. -d2
is now enough for cc-graph.sh to work.

Also, this new technique *should not* require crazy --min-parallelism
options. If you don't mind, would you run a scan without that?

> All the outputs ended with similar lines (but not the same hosts):
>
>    c.d.52.164: 0/1/0/0/0/3 2048.00/75/0 116015/-1/-1
>    c.d.52.165: 0/1/0/0/0/3 2048.00/__EOF__

Did all the scans say 2048.00? I wouldn't expect the congestion window
to grow that big except in the 2048-2048 case.

> Also, I love you cc-graph.sh script but it doesn't scale to huge scans.  It
> errors out with:
>
> awk: cmd. line:1: (FILENAME=david_new_1a.nmap FNR=1916551) fatal: print to
> "standard output" failed (No space left on device)
> flexmap/nmap/testing-new/cc-graph.sh: line 16: echo: write error: No space
> left on device
> awk: cmd. line:1: (FILENAME=david_new_1a.nmap FNR=78162) fatal: print to
> "standard output" failed (No space left on device)
> libplot error: output stream jammed
> graph: error: could not close graphing device
> Graph saved to ./david_new_1a.svg.
>
> I suspect that it is trying to write too much to 'one line' of stdout.  I'm
> not an awk ninja like you but I'm pretty sure I've deciphered what you're
> doing with awk so if you want me to convert it to a short perl script let
> me know.

Sure, that would be nice, but it's not urgent. I haven't run into the
problem you describe. That's what I get for assuming no one else would
use my quick hack :)

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org