Re: [RFC] Crazy idea for deferring hosts in progress to a later hostgroup

[Jacek Wielemborek <d33tah () gmail com>]

W dniu 26.03.2015 o 16:30, Daniel Miller pisze:
> List,
>
> This is a crazy idea, but I think it might work. First, the problem: Nmap
> (portscan phase) scans hosts in a large group (hostgroup), and while some
> hosts may finish quickly, a few slow systems may drag on for a long time,
> preventing new hosts from being scanned until they are finished.
>
> I propose that these slow hosts be identified and "frozen," then added to
> the next hostgroup to be scanned. This way, new (potentially fast) hosts
> can be scanned while these slow ones are finishing.

Hi Daniel,

Why is this crazy? I like this idea! This is much simpler than the
concept of pipelining and could actually give a huge speedup. Given that
I think that I know scan_engine.cc quite a bit already, here are my
comments:

> Here are the challenges I see to this approach:
>
> 1. The data structure for a "host being scanned" is the HostScanStats
> (HSS), which has a UltraScanInfo (USI) as a data member. This USI is
> different for each run of the ultrascan engine (hostgroup), so the
> "freezing" and "thawing" would have to be able to replace this and any
> related data members without losing results or timing info.

Perhaps just killing the scan of this slow host and re-starting it in
the new group would be enough?

> 2. Hostgroups are defined not only by strict numbers (e.g. we have 1024 new
> hosts, that's one hostgroup), but also by other factors like connectedness
> (e.g. a directly-connected IP will be put into a different hostgroup from
> those that require IP routing). So if a slow host is deferred, the next
> hostgroup to be scanned may not be appropriate for it to be in.

How about always creating the "deferred" hostgroup that would be scanned
last and has no other kinds of hosts there?

> 3. Multiple scan types can be used in one scan (e.g. -sSU) which is
> implemented as subsequent calls to ultrascan. If a HSS is deferred in the
> SYN scan phase, should its Target be started in the UDP scan phase, or
> should the entire Target be deferred to the next hostgroup? I tend to think
> it should be allowed to start the UDP scan phase, but this would have to be
> engineered.

I'd say that if the host is slow in one type of scan, he's likely to be
in others, but I'm not sure.

> 4. What determines a "slow host"? Should there be a numeric limit on how
> many can be deferred? On how many deferrals can happen for one host?
>
> I'd really like to hear your feedback on this idea. Thanks!
>
> Dan

I think that we could detect a slow host by comparing the cwnds of all
hosts. If one of them drastically stands out in the negative sense, it
sounds like a good candidate to defer.

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Sent through the dev mailing list
https://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/