[BUG] NSE/Nsock filehandle exhaustion

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Developers,

I hate to be reporting a bug without a patch but I haven't been able to
fully track this one down and I'm sure someone here is going to have more
insight into the problem than me.

With the latest NSE implementation compiled from SVN, Nmap runs my machine
out of filehandles when I scan large block of machines at a time.

Here is sample output:

...
SCRIPT ENGINE: Will run ././nmap/current/scripts//ripeQuery.nse against
132.239.74.211
SCRIPT ENGINE: Running scripts.
SCRIPT ENGINE: Runlevel: 1.000000
Initiating SCRIPT ENGINE at 23:49
SCRIPT ENGINE Timing: About 0.00% done
Socket troubles: Too many open files
Socket troubles: Too many open files
...lots of socket trouble errors...
Socket troubles: Too many open files
Segmentation fault


Over 1024 copies of the ripeQuery.nse script were being executed in that
hostgroup.  I went ahead and increased the max filehandle count with ulimit
- -n and /etc/security/limits.conf from 1024 to 10240.  Unfortunately instead
of solving the problem, it hits another:

SCRIPT ENGINE: Will run ././nmap/current/scripts//ripeQuery.nse against
132.239.75.10
SCRIPT ENGINE: Running scripts.
SCRIPT ENGINE: Runlevel: 1.000000
Initiating SCRIPT ENGINE at 23:58
SCRIPT ENGINE Timing: About 0.00% done
nmap: gh_list.c:346: gh_list_remove_elem: Assertion `list->count != 0 ||
(list->first == ((void *)0) && list->last == ((void *)0))' failed. Aborted

I've done some digging and the issue seems to be the number of concurrent
sockets that are being opened.

To test, I modified the ripeQuery.nse script to just be:

action = function(host, port)
        local socket = nmap.new_socket()
        socket:connect("whois.ripe.net", 43)
        socket:close()
        socket = nil
        return
end

The idea here is to make a new Nsock socket, connect, and then close.
I understand that if there are too many sockets open simultaneously there
will be a problem.  By reducing the script to just this, I was trying to
ensure that Nmap would be closing sockets fast enough that no more than 1024
wouldn't be open at once.  The problem still shows up just as quickly before
though.

I then suspected that there was a Lua garbage collection problem so I added
socket = nil after the close but that also didn't solve the problem.  I've
looked at the Nsock close code and it does seem to be properly destroying
the socket filehandle.

I've commented out the connect portion of the script and the problem
goes away. That is, make a new socket, close (Nsock close doesn't do
anything on an unconnected socket) and then return.  This says to me that
when the problem is occurring, the connect() is actually being called more
than 1024 times more than close().

Is it possible that Nmap/NSE is calling socket:connect() more than 1024
times in parallel *before* the parallel scripts get to the socket:close()
call?  This doesn't sound very likely to me.

How do I go about troubleshooting this?  I'd like some way of seeing the
number of simultanious scripts being blocked waiting for the connect call
to see if it gets over 1024.

Maybe there needs to be some (tunable) cap on the number parallel NSE
scripts or number of sockets allowed open by NSE at a time.  I'm thinking
that nmap.new_socket() could block if the number of connected/open sockets
goes above some threshold.

Please let me know if there is anything I can do to help troubleshoot this
or if I need to clarify anything stated above.

Brandon

- -- 
Brandon Enright
Network Security Analyst
UCSD Network Operations
bmenrigh () ucsd edu


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFG03JpqaGPzAsl94IRAvtbAKCV0Q0ejdWuZgcaSB+jfbSkQ8JyigCfb1z0
M2G/h4YS2YlG5tqyymKE6SI=
=XOfS
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org