Thread Parallelism for Sockets (Fix for infinite loops/deadlocks in NSE)

["Patrick Donnelly" <batrick.donnelly () gmail com>]

There was a problem found in NSE by Alex Jurkiewicz [1] that caused an
infinite loop (actually a deadlock) in NSE when many scripts tried to
open more than one socket. Particularly, showOwner.nse would open a
socket for both the service and the identification (113) ports.
Because NSE has a limit of 10 open sockets for threads, the system
would deadlock and no further progress would be made.

I've created a fix (attached) that allows up to 10 threads (can be
changed with --max-parallelism, this is used elsewhere for "unrelated"
reasons, I wonder if we should have a different option?) to have any
number of sockets open (connected). You can (and please do) test the
change in my branch at svn://svn.insecure.org/nmap-exp/patrick/inf

Some technical details of the implementation:

1) The open sockets are paired with a per thread unique userdata
(henceforth called 'proxies') in a weak keyed table. When all sockets
have closed or been collected, the userdata is collected and a slot is
freed in a fully weak table with thread proxy pairs (which contains
the number of threads with open sockets). When the proxy is collected
(and thus a slot freed), a thread waiting to open a socket is moved to
running and given a lock.

2) Previously, when a thread tried to connect when there were too many
sockets open, a handler for connect would yield the thread and do some
pretty hackish things to basically yield a thread from C without
returning:

In Lua:
function connect_handler(...)
   if cond then coroutine.yield() end
   return connect(...)
end

This is dangerous so I've changed the connect handler to actual Lua
code that does this.

Please post any comments or problems experienced here.

[1] http://seclists.org/nmap-dev/2008/q1/0364.html

Cheers,

-- 
-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing
to do and always a clever thing to say."

-Will Durant

Attachment: nse_inf.patch
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org