Re: [nmap-svn] r12765 - nmap/docs

[Patrick Donnelly <batrick () batbytes com>]

Hi Fyodor,

On Mon, Mar 30, 2009 at 6:47 PM,  <commit-mailer () insecure org> wrote:
> Author: fyodor
> Date: Mon Mar 30 17:47:50 2009
> New Revision: 12765
>
> Modified:
>   nmap/docs/TODO
>
> Log:
> TODO after talking with David
>
> Modified: nmap/docs/TODO
> ==============================================================================
> [...]
> +o Optimize NSE Performance--e.g. measure the current performance and
> +  see what can be improved in terms of scheduling scan threads,
> +  determining how many to run concurrently, looking at CPU load items,
> +  etc.

It is my belief that NSE has reached a point where it no longer needs
much if any look at performance. The current system will begin by
running all threads (which could be thousands) but will finish this
process rapidly. Nearly all threads will end up in a waiting queue and
only --max-parallelism (usually 10) scripts will run at any time. Once
NSE reaches this stable state of running only --max-parallelism
scripts, the majority of execution time is used by the scripts
themselves (Except with the new garbage collection scheme which can be
removed once the nsock library is fixed).

I believe for NSE performance (that is the length of the scan, not the
engine itself) to be improved, we need to look at better communication
between scripts that do the same thing (reading a banner in
particular). To do this, we would be best served by using a library
that caches results so threads needn't use sockets at all to get data
that are already known. My next task for nmap that I will begin work
on soon is improving the http library to cache data so scripts are no
longer repeatedly querying the same information. I believe the comm
library would also benefit.

> +o Consider whether we should include some sort of NSE debugger.  Or we
> +  could include something simpler.  For example, some developers (such
> +  as Ron) already make use of Patrick's traceback.nse in their
> +  experimental trees.

One of the nice things about the new Lua script engine is a developer
can _very_ easily change the engine to include new debugging output
without recompiling the nmap. I think there is room for more standard
debugging info (such as outputting a stack trace when a script
finishes) to be added throughout the NSE scan but I think we should
encourage developers to add their own personalized debugging output
directly to the engine.

-- 
-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing
to do and always a clever thing to say."

-Will Durant

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org