Re: [PATCH] Save space/time: completedHosts "lifetimes"

[David Fifield <david () bamsoftware com>]

On Wed, Sep 17, 2008 at 06:42:38PM -0500, Kris Katterjohn wrote:
> I've been thinking about the completedHosts list used in ultra_scan.  It's
> used to hold the hosts which have completed in case a late reply comes.
>
> But if you're doing a larger scan that ends up being quite slow overall, but
> with hosts completing throughout, keeping these completed hosts wastes quite a
> lot of space.  An example of this situation is a no-ping SYN scan with a large
> hostgroup (so a single ultra_scan run is slow).  And there's also the fact
> that this list can be traversed (after incompleteHosts) when searching to see
> if a received packet is for us or not.  When doing a large scan, Nmap doesn't
> have a very strict pcap filter, so there is a potential for the oft traversal
> of this large list for essentially no gain.
>
> It doesn't even need to be that massive of a scan to have completed hosts
> waiting for a while in this list.  I tested with -T3 and -T4, and had hosts
> completing throughout the ultra_scan run.  Any completed hosts would be
> sitting for a long time using the slower timing options.
>
> So I propose (patch attached) limiting the lifetime of these hosts to the
> standard TCP MSL (two minutes).  This should be ample time for any late
> replies, especially since this is after the normal wait period (which is why
> the host is classified as "completed").  I would like a shorter time, but I
> think this value is a good median for making sure there's more than enough
> time for a reply, and still removing hosts from the list often.  I make sure
> not to remove a host if it's the one used for port-scan pings, since I don't
> think anything stops it from entering this list.

Sorry I didn't look at this patch sooner. I see that you have already
committed it. It looks good and simple, I didn't find any obvious
problems.

Fyodor, I remember you mentioned UDP scans with a big host group size
taking gigs of RAM. I debugged for a while looking for a memory leak but
didn't find anything. Maybe the ever-growing completedHosts list was the
culprit?

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org