Re: Weird Crash - "WAITING_TO_RUNNING"

[Patrick Donnelly <batrick () batbytes com>]

On Fri, Mar 18, 2011 at 7:16 PM, Nathan <nathan.stocks () gmail com> wrote:
> Ok, I've got some results with 5.51 on the same box.  Same command as
> before:
>
> time /usr/bin/sudo ./nmap --datadir . -sS -sV -T4 -p 1-65535 74.62.92.70 -P0
> -v
>
> (compiled in place - didn't run 'make install')
>
> Elapsed time: 15m13s
> Memory usage: 99MB
>
> (same compile, but omitting '--datadir .' on the command, we get pretty much
> the same thing)
>
> Elapsed time: 15m21s
> Memory usage: 99MB
>
> Now here's the interesting thing, sticking to the original command, if I
> edit the nse_main.lua in the local directory and set CONCURRENCY_LIMIT=100,
> like you had it when we were testing your patches, we get this:
>
> Elapsed time: 14m16s
> Memory usage: 74MB
>
> Fascinating, eh?  If you bring the concurrent threads down, you get a lower
> peak memory usage AND faster total run time.  Well, that makes sense,
> there's only 8 cores on this box, after all.  Looks like the concurrency was
> too high and we were wasting some time and memory just managing all the
> stuff in flight.  So what would happen if we brought it even closer?  Here's
> what I got when I set CONCURRENCY_LIMIT=10.

NSE doesn't use operating system threads. NSE uses coroutines which
are referred to as threads by Lua.

> Elapsed time: 21m31s
> Memory usage: 67MB
>
> Oops!  Even though we continued to trim peak memory usage, our run time just
> got slaughtered.  Obviously the scripts aren't using that much processor
> time, so we just artifically bottlenecked ourselves with a concurrency limit
> set too low.

Yup.

> The nice thing for me is that any way I take it, 5.51 isn't crashing and
> doesn't necessarily need any monkey-patching from me.

Isn't it still taking much longer than some of our earlier tests?

-- 
- Patrick Donnelly
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/