Re: Suspect that --host-timeout is not working in 4.50?

[Randolph Reitz <rreitz () fnal gov>]

On Dec 14, 2007, at 4:42 PM, jah wrote:

> On 14/12/2007 20:52, Randolph Reitz wrote:
> > Hi,
> >
> > I have installed nmap 4.50 on the scanner farm here at Fermilab and
> > I've noticed that some nmap scans are running a long time.  For
> > example ...
> >
> > scanner   5311 31009  0 12:17 ?        00:00:00 /bin/bash ./bin/
> > run_nmap.sh --pro -d 1 -sS -p 1-65535 -A 131.225.232.A 131.225.232.B
> > 131.225.232.C 131.225.232.D
> > root      5319  5311  2 12:17 ?        00:03:10 /usr/local/bin/nmap -
> > sS -p 1-65535 -P0 -T4 --osscan-limit --osscan-guess --host-timeout  
> > 15m
> > -A -oX - 131.225.232.D
> >
> > It's now
> > date
> > Fri Dec 14 14:47:47 CST 2007
> >
> > The nmap started at 12:17 and has collected 3 minutes of CPU so far.
> > The host_timeout is set for 15 minutes.  So far, I've collected
> > hundreds of examples of long-running nmap scans.  However, I've
> > noticed that nmap 4.50 is much faster than 4.2.
> >
> > Does anyone else have a problem with --host-timeout?
> Hello Randolph,
>
> I don't seem to be having any problems with --host-timeout, may I  
> propose a quick test...
>
> Perform a simple test scan against a couple of hosts with the aim of  
> finding a host/scan combination that takes at least 2 seconds, but  
> as short as possible (this is supposed to be a quick test).  An  
> example might be:
>
> nmap -d -sU -p1-5000 <target>
>
> When you have a total scan time that suits, add the lowest  
> permissible host-timeout (1501ms):
>
> nmap -d -sU -p1-5000 --host-timeout 1501 <target>
>
> if host-timeout is working properly, you should see something like:
>
> ...
> Completed ARP Ping Scan at 22:35, 0.05s elapsed (1 total hosts)
> ...
> <target> timed out during UDP Scan (0 hosts left)
> Completed UDP Scan at 22:35, 1.46s elapsed (1 host timed out)
> Host <target> appears to be up ... good.
> Skipping host <target> due to host timeout
> ...
>
> If that's a success, you could start building up the scan paramaters  
> again and hopefully determine what's gone wrong.
>
> Hope that helps a bit,
>
> jah

Thanks for your reply.  The expected response was "Hey, it's open  
source, so go fix it!".

Quick answer, it's the script engine that seems to be ignoring the  
host_timeout option.

I have roughly 12,000 hosts to choose from, I'll just pluck one from a  
log file of systems that took a long time to port scan earlier today.   
I can get 4.50 to timeout with the options you suggest.  For example...

[scanner@clouseau ~]$ nmap -d -sS -p 1-5000 --host_timeout 10s   
131.225.136.140
host-timeout is given in milliseconds, so you specified less than 15  
seconds (10000ms). This is allowed but not recommended.

Starting Nmap 4.50 ( http://insecure.org ) at 2007-12-14 17:51 CST
--------------- Timing report ---------------
   hostgroups: min 1, max 100000
   rtt-timeouts: init 1000, min 100, max 10000
   max-scan-delay: TCP 1000, UDP 1000
   parallelism: min 0, max 0
   max-retries: 10, host-timeout: 10000
---------------------------------------------
Initiating Ping Scan at 17:51
Scanning 131.225.136.140 [2 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or ((tcp or udp) and (src host 131.225.136.140)))
We got a TCP ping packet back from 131.225.136.140 port 80 (trynum = 0)
Completed Ping Scan at 17:51, 0.01s elapsed (1 total hosts)
mass_rdns: Using DNS server 131.225.8.120
mass_rdns: Using DNS server 131.225.17.150
Initiating Parallel DNS resolution of 1 host. at 17:51
mass_rdns: 0.00s 0/1 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1]
Completed Parallel DNS resolution of 1 host. at 17:51, 0.00s elapsed
DNS resolution of 1 IPs took 0.00s. Mode: Async [#: 2, OK: 1, NX: 0,  
DR: 0, SF: 0, TR: 1, CN: 0]
Initiating SYN Stealth Scan at 17:51
Scanning plainwell.fnal.gov (131.225.136.140) [5000 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or (tcp and (src host 131.225.136.140)))
Discovered open port 21/tcp on 131.225.136.140
Discovered open port 23/tcp on 131.225.136.140
Discovered open port 4045/tcp on 131.225.136.140
Increased max_successful_tryno for 131.225.136.140 to 1 (packet drop)
Increasing send delay for 131.225.136.140 from 0 to 5 due to 18 out of  
60 dropped probes since last increase.
Increased max_successful_tryno for 131.225.136.140 to 2 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 3 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 4 (packet drop)
Increasing send delay for 131.225.136.140 from 5 to 10 due to  
max_successful_tryno increase to 4
Increased max_successful_tryno for 131.225.136.140 to 5 (packet drop)
Increasing send delay for 131.225.136.140 from 10 to 20 due to  
max_successful_tryno increase to 5
131.225.136.140 timed out during SYN Stealth Scan (0 hosts left)
Completed SYN Stealth Scan at 17:51, 10.00s elapsed (1 host timed out)
Host plainwell.fnal.gov (131.225.136.140) appears to be up ... good.
Skipping host plainwell.fnal.gov (131.225.136.140) due to host timeout
Final times for host: srtt: 376 rttvar: 51  to: 100000

Read from /usr/local/share/nmap: nmap-services.
Nmap done: 1 IP address (1 host up) scanned in 10.055 seconds
            Raw packets sent: 506 (22.244KB) | Rcvd: 306 (14.076KB)

If I use all ports...

[scanner@clouseau ~]$ nmap -d -sS -p 1-65535 --host_timeout 10s   
131.225.136.140
host-timeout is given in milliseconds, so you specified less than 15  
seconds (10000ms). This is allowed but not recommended.

Starting Nmap 4.50 ( http://insecure.org ) at 2007-12-14 17:52 CST
--------------- Timing report ---------------
   hostgroups: min 1, max 100000
   rtt-timeouts: init 1000, min 100, max 10000
   max-scan-delay: TCP 1000, UDP 1000
   parallelism: min 0, max 0
   max-retries: 10, host-timeout: 10000
---------------------------------------------
Initiating Ping Scan at 17:52
Scanning 131.225.136.140 [2 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or ((tcp or udp) and (src host 131.225.136.140)))
We got a TCP ping packet back from 131.225.136.140 port 80 (trynum = 0)
Completed Ping Scan at 17:52, 0.01s elapsed (1 total hosts)
mass_rdns: Using DNS server 131.225.8.120
mass_rdns: Using DNS server 131.225.17.150
Initiating Parallel DNS resolution of 1 host. at 17:52
mass_rdns: 0.00s 0/1 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1]
Completed Parallel DNS resolution of 1 host. at 17:52, 0.00s elapsed
DNS resolution of 1 IPs took 0.00s. Mode: Async [#: 2, OK: 1, NX: 0,  
DR: 0, SF: 0, TR: 1, CN: 0]
Initiating SYN Stealth Scan at 17:52
Scanning plainwell.fnal.gov (131.225.136.140) [65535 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or (tcp and (src host 131.225.136.140)))
Discovered open port 21/tcp on 131.225.136.140
Discovered open port 23/tcp on 131.225.136.140
Increased max_successful_tryno for 131.225.136.140 to 1 (packet drop)
Increasing send delay for 131.225.136.140 from 0 to 5 due to 18 out of  
59 dropped probes since last increase.
Increased max_successful_tryno for 131.225.136.140 to 2 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 3 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 4 (packet drop)
Increasing send delay for 131.225.136.140 from 5 to 10 due to  
max_successful_tryno increase to 4
Increased max_successful_tryno for 131.225.136.140 to 5 (packet drop)
Increasing send delay for 131.225.136.140 from 10 to 20 due to  
max_successful_tryno increase to 5
131.225.136.140 timed out during SYN Stealth Scan (0 hosts left)
Completed SYN Stealth Scan at 17:52, 10.00s elapsed (1 host timed out)
Host plainwell.fnal.gov (131.225.136.140) appears to be up ... good.
Skipping host plainwell.fnal.gov (131.225.136.140) due to host timeout
Final times for host: srtt: 359 rttvar: 27  to: 100000

Read from /usr/local/share/nmap: nmap-services.
Nmap done: 1 IP address (1 host up) scanned in 10.118 seconds
            Raw packets sent: 505 (22.200KB) | Rcvd: 305 (14.030KB)

The host-timeout works!  However, if I add service detection (and I  
bumped the host-timeout to 1m)...

[scanner@clouseau ~]$ nmap -d -sS -p 1-65535 --host_timeout 1m  -A  
131.225.136.140

Starting Nmap 4.50 ( http://insecure.org ) at 2007-12-14 18:13 CST
--------------- Timing report ---------------
   hostgroups: min 1, max 100000
   rtt-timeouts: init 1000, min 100, max 10000
   max-scan-delay: TCP 1000, UDP 1000
   parallelism: min 0, max 0
   max-retries: 10, host-timeout: 60000
---------------------------------------------
Initiating Ping Scan at 18:13
Scanning 131.225.136.140 [2 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or ((tcp or udp) and (src host 131.225.136.140)))
We got a TCP ping packet back from 131.225.136.140 port 80 (trynum = 0)
Completed Ping Scan at 18:13, 0.01s elapsed (1 total hosts)
mass_rdns: Using DNS server 131.225.8.120
mass_rdns: Using DNS server 131.225.17.150
Initiating Parallel DNS resolution of 1 host. at 18:13
mass_rdns: 0.00s 0/1 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1]
Completed Parallel DNS resolution of 1 host. at 18:13, 0.00s elapsed
DNS resolution of 1 IPs took 0.00s. Mode: Async [#: 2, OK: 1, NX: 0,  
DR: 0, SF: 0, TR: 1, CN: 0]
Initiating SYN Stealth Scan at 18:13
Scanning plainwell.fnal.gov (131.225.136.140) [65535 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or (tcp and (src host 131.225.136.140)))
Discovered open port 21/tcp on 131.225.136.140
Discovered open port 23/tcp on 131.225.136.140
Increased max_successful_tryno for 131.225.136.140 to 1 (packet drop)
Increasing send delay for 131.225.136.140 from 0 to 5 due to 18 out of  
59 dropped probes since last increase.
Increased max_successful_tryno for 131.225.136.140 to 2 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 3 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 4 (packet drop)
Increasing send delay for 131.225.136.140 from 5 to 10 due to  
max_successful_tryno increase to 4
Increased max_successful_tryno for 131.225.136.140 to 5 (packet drop)
Increasing send delay for 131.225.136.140 from 10 to 20 due to  
max_successful_tryno increase to 5
SYN Stealth Scan Timing: About 1.73% done; ETC: 18:42 (0:28:25  
remaining)
131.225.136.140 timed out during SYN Stealth Scan (0 hosts left)
Completed SYN Stealth Scan at 18:14, 60.00s elapsed (1 host timed out)
Initiating Service scan at 18:14
Initiating Traceroute at 18:14
131.225.136.140: hop distance parameters -> hg:64 ttl:59
131.225.136.140: guessing hop distance at 5
Completed Traceroute at 18:14, 0.00s elapsed
Initiating Parallel DNS resolution of 7 hosts. at 18:14
mass_rdns: 0.00s 0/5 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 5]
Completed Parallel DNS resolution of 7 hosts. at 18:14, 0.00s elapsed
DNS resolution of 5 IPs took 0.00s. Mode: Async [#: 2, OK: 5, NX: 0,  
DR: 0, SF: 0, TR: 5, CN: 0]
SCRIPT ENGINE: Initiating script scanning.
SCRIPT ENGINE: Script scanning plainwell.fnal.gov.
SCRIPT ENGINE: Using /usr/local/libexec/nmap/nselib-bin/?.so;./?.so;/ 
usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so to search  
for C-modules and /usr/local/share/nmap/nselib/?.lua;./?.lua;/usr/ 
local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/ 
local/lib/lua/5.1/?.lua;/usr/local/lib/lua/5.1/?/init.lua for Lua- 
modules
SCRIPT ENGINE: Initialized 21 rules
SCRIPT ENGINE: Matching rules.
SCRIPT ENGINE: Will run /usr/local/share/nmap/scripts/anonFTP.nse  
against 131.225.136.140:21
SCRIPT ENGINE: Will run /usr/local/share/nmap/scripts/bruteTelnet.nse  
against 131.225.136.140:23
SCRIPT ENGINE: Running scripts.
SCRIPT ENGINE: Runlevel: 1.000000
Initiating SCRIPT ENGINE at 18:14
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:15 (0:00:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:16 (0:01:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:17 (0:01:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:18 (0:02:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:19 (0:02:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:20 (0:03:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:21 (0:03:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:22 (0:04:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:23 (0:04:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:24 (0:05:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:25 (0:05:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:26 (0:06:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:27 (0:06:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:28 (0:07:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:29 (0:07:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:30 (0:08:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:31 (0:08:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:32 (0:09:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:33 (0:09:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:34 (0:10:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:35 (0:10:30 remaining)

I killed it.  It's now 18:25, so the nmap has been running for ~13  
minutes.  If I drop the greedy -p 1-65535 and go for -F ...

[scanner@clouseau ~]$ nmap -d -sS -F --host_timeout 1m  -A  
131.225.136.140

Starting Nmap 4.50 ( http://insecure.org ) at 2007-12-14 18:27 CST
--------------- Timing report ---------------
   hostgroups: min 1, max 100000
   rtt-timeouts: init 1000, min 100, max 10000
   max-scan-delay: TCP 1000, UDP 1000
   parallelism: min 0, max 0
   max-retries: 10, host-timeout: 60000
---------------------------------------------
Initiating Ping Scan at 18:27
Scanning 131.225.136.140 [2 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or ((tcp or udp) and (src host 131.225.136.140)))
We got a TCP ping packet back from 131.225.136.140 port 80 (trynum = 0)
Completed Ping Scan at 18:27, 0.00s elapsed (1 total hosts)
mass_rdns: Using DNS server 131.225.8.120
mass_rdns: Using DNS server 131.225.17.150
Initiating Parallel DNS resolution of 1 host. at 18:27
mass_rdns: 0.00s 0/1 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1]
Completed Parallel DNS resolution of 1 host. at 18:27, 0.00s elapsed
DNS resolution of 1 IPs took 0.00s. Mode: Async [#: 2, OK: 1, NX: 0,  
DR: 0, SF: 0, TR: 1, CN: 0]
Initiating SYN Stealth Scan at 18:27
Scanning plainwell.fnal.gov (131.225.136.140) [1272 ports]
Packet capture filter (device eth0): dst host 131.225.12.197 and (icmp  
or (tcp and (src host 131.225.136.140)))
<open ports discovered>
Increased max_successful_tryno for 131.225.136.140 to 1 (packet drop)
Increasing send delay for 131.225.136.140 from 0 to 5 due to 18 out of  
60 dropped probes since last increase.
Increased max_successful_tryno for 131.225.136.140 to 2 (packet drop)
Increased max_successful_tryno for 131.225.136.140 to 3 (packet drop)
Discovered open port 32780/tcp on 131.225.136.140
Increased max_successful_tryno for 131.225.136.140 to 4 (packet drop)
Increasing send delay for 131.225.136.140 from 5 to 10 due to  
max_successful_tryno increase to 4
Discovered open port 32778/tcp on 131.225.136.140
Increased max_successful_tryno for 131.225.136.140 to 5 (packet drop)
Increasing send delay for 131.225.136.140 from 10 to 20 due to  
max_successful_tryno increase to 5
<open ports discovered>
Completed SYN Stealth Scan at 18:28, 34.51s elapsed (1272 total ports)
Initiating Service scan at 18:28
Scanning 15 services on plainwell.fnal.gov (131.225.136.140)
Got nsock CONNECT response with status TIMEOUT - aborting this service
Completed Service scan at 18:28, 26.01s elapsed (1 host timed out)
Initiating Traceroute at 18:28
131.225.136.140: hop distance parameters -> hg:64 ttl:59
131.225.136.140: guessing hop distance at 5
Completed Traceroute at 18:28, 0.01s elapsed
Initiating Parallel DNS resolution of 7 hosts. at 18:28
mass_rdns: 0.00s 0/5 [#: 2, OK: 0, NX: 0, DR: 0, SF: 0, TR: 5]
Completed Parallel DNS resolution of 7 hosts. at 18:28, 2.50s elapsed
DNS resolution of 5 IPs took 2.50s. Mode: Async [#: 2, OK: 5, NX: 0,  
DR: 0, SF: 0, TR: 6, CN: 0]
SCRIPT ENGINE: Initiating script scanning.
SCRIPT ENGINE: Script scanning plainwell.fnal.gov.
SCRIPT ENGINE: Using /usr/local/libexec/nmap/nselib-bin/?.so;./?.so;/ 
usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so to search  
for C-modules and /usr/local/share/nmap/nselib/?.lua;./?.lua;/usr/ 
local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/ 
local/lib/lua/5.1/?.lua;/usr/local/lib/lua/5.1/?/init.lua for Lua- 
modules
SCRIPT ENGINE: Initialized 21 rules
SCRIPT ENGINE: Matching rules.
SCRIPT ENGINE: Will run /usr/local/share/nmap/scripts/anonFTP.nse  
against 131.225.136.140:21
SCRIPT ENGINE: Will run /usr/local/share/nmap/scripts/bruteTelnet.nse  
against 131.225.136.140:23
SCRIPT ENGINE: Running scripts.
SCRIPT ENGINE: Runlevel: 1.000000
Initiating SCRIPT ENGINE at 18:28
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:29 (0:00:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:30 (0:01:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:31 (0:01:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:32 (0:02:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:33 (0:02:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:34 (0:03:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:35 (0:03:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:36 (0:04:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:37 (0:04:30 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:38 (0:05:00 remaining)
SCRIPT ENGINE Timing: About 50.00% done; ETC: 18:39 (0:05:30 remaining)

Same problem.  Once the script engine starts, the host-timeout seems  
to be ignored.

Thanks,
Randy Reitz
Fermilab




_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org