Re: NSE: odd output, testing, etc

["Patrick Donnelly" <batrick.donnelly () gmail com>]

Brandon, Ron, and I have been discussing these problems privately by
email to track the causes. Here are the results:

> * NSE is overly aggressive with parallelism.  It isn't unusual for NSE
>  to report more than 2000 active NSE scripts.  When this happens Lua
>  seems to thrash and NSE scanning slows to a crawl.  I think this has
>  the ability to trigger the "lock, (null), <int>, tcp, ERROR" errors
>  describe below.

This shouldn't be a problem as the default of 10 script threads may
use network resources, all the other 1990 threads are blocked. Memory
is also not a concern as each thread only consumes about 1 KB of
memory. NSE shouldn't slow to a crawl. I assume most of the scripts
were blocked on a socket. I haven't observed any problems with NSE
encountering a deadlock or infinite loop since the problems discussed
in [1][2].

> * There seems to be some sort of script deadlocking detection that can
>  output "SCRIPT ENGINE: lock".  It isn't clear what circumstances are
>  required to cause this but I'm not convinced it is always a real
>  deadlock.

This was caused by a script timing out and the top item on the stack
being printed. The engine set the state for a timed out thread to
LUA_ERRRUN so that later on the engine would print out the top item on
the stack which could be anything. In the case of "lock", it was a
thread blocked on mutex("lock").

> * It seems a script with a handle to a mutex won't release it if the
>  script crashes (causing a deadlock).

This is avoidable depending on the nature of the crash. Basically, you
can use (x)pcall Lua functions to make a protected call so you can
unlock any mutexes you have. However, there is a problem that you
cannot use a pcall if, in the function you are calling, you will be
using any functions that can yield the thread (e.g. nsock or mutex
functions). This is because you cannot yield a thread across a C Call
boundary (pcall). There is a solution by using coroutines to naturally
catch errors (because coroutine.resume does not propagate the thread
error to the caller) or if NSE yielded the thread then yield in the
caller. Essentially:

local co = coroutine.create(function() --[[ do stuff with sockets ]] end)
while true do
  local status, result = coroutine.resume(co)
  if not status then -- the thread threw an error
    -- do stuff
    break;
  elseif coroutine.status(co) == "suspended" then
   coroutine.yield() -- yield in caller
  else
    -- the thread finished naturally
    break;
  end
end

Unfortunately you cannot use coroutines for this purpose yet. I am
developing a patch so that you can.

> * Sometimes the script engine will print a series of "SCRIPT ENGINE:
>  (null)" right before the engine completes.
>
> * Sometimes a script will exit and the only output is "SCRIPT ENGINE:
>  tcp".
>
> * Sometimes a script will exit and the only output is "SCRIPT ENGINE:
>  ERROR".
>
> * Sometimes a script will exit and the only output is "SCRIPT ENGINE:
>  <int>" where <int> is typically a small number.  One such example is
>  "SCRIPT ENGINE: 4".

These are all the same problem as the "SCRIPT_ENGINE: lock" as above.

[1] http://seclists.org/nmap-dev/2008/q1/0364.html
[2] http://seclists.org/nmap-dev/2008/q3/0130.html

Cheers,

-- 
-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing
to do and always a clever thing to say."

-Will Durant

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org