Re: NSE: odd output, testing, etc

[David Fifield <david () bamsoftware com>]

On Sun, Dec 21, 2008 at 05:10:59AM -0700, Patrick Donnelly wrote:
> Brandon, Ron, and I have been discussing these problems privately by
> email to track the causes. Here are the results:
>
> > * NSE is overly aggressive with parallelism.  It isn't unusual for NSE
> >  to report more than 2000 active NSE scripts.  When this happens Lua
> >  seems to thrash and NSE scanning slows to a crawl.  I think this has
> >  the ability to trigger the "lock, (null), <int>, tcp, ERROR" errors
> >  describe below.
>
> This shouldn't be a problem as the default of 10 script threads may
> use network resources, all the other 1990 threads are blocked. Memory
> is also not a concern as each thread only consumes about 1 KB of
> memory. NSE shouldn't slow to a crawl. I assume most of the scripts
> were blocked on a socket. I haven't observed any problems with NSE
> encountering a deadlock or infinite loop since the problems discussed
> in [1][2].

So it sounds like the scripts are running and blocking fine. The only
problem is that they tend to time out before they get a chance to do
anything. Brandon said that after one round of scanning with more than
300 script threads running, in the next round NSE locks immediately.
Could something about timing out scripts be corrupting the Lua state?
The problem goes away when r11122 is reverted. That was the change that
made use of a persistent Lua state.

Maybe resources held by timed-out threads not being garbage collected?

If worse comes to worst and we can't solve the problem, I'll take out
r11122 and we'll live with a non-persistent registry for another
release. It would be better to keep the persistent registry and solve
the parallelism problem though.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org