NSE Infrastructure Suggestions

[Gaveen Prabhasara <gaveens () gmail com>]

Hi,

I have given some thought about the rapidly developing Nmap scripting
system. There's a few things I'd like to draw your attention to
regarding the organization of NSE. There's a fairly decent structure
of things now, but I'd like to suggest these to ease up the lives of
script writers and Nmap maintainers. 

These are things I practically came across when planning some scripts.
most of the current scripts should not be affected, should we opt for
these changes. I send this rather lengthy email in hope that this 
would help to make Nmap more friendly to people who actually use it. 


* Dependency field instead of runlevel field

First thing I like to mention is the current concept of runlevels
can be altered into a dependency field to achieve ease for the script
developer. Runlevel field is a good concept and could be used to
achieve efficient results. But in the long run it will create
usability issues. With a fairly large script set, it will be hard to
determine the appropriate runlevel for a new script. And there will
be situations where we would want to put newer scripts in higher
run levels. As you can see the situation could get easily out of
hand given a respectable amount of scripts. I'm sure you are going to
agree that the script collection is growing.

As a solution I suggest we get rid of runlevel concept and introduce
a dependency field. By doing so we will take the the burden of the
complex decision "when to run" out of the script developer. She will
only have to worry about what data she needs to fetch before the
execution of her script. Meanwhile NSE can take care of the order of
execution. May be this can be integrated into the script DB as a one
time process when the --script-updatedb is executed.


* Knowledge Base instead of the Registry

KB <---> NSE <---> Scripts

I'm not expecting to immediately available, but with a KB and the
concept of dependencies we would be greatly organizing the things
in NSE. While the registry is also a good implementation, it also
is a simple solution. I believe be can extract a lot more out of
the new found scripting ability in Nmap, if we plan a little more
on capturing and managing the knowledge. A KB is much similar to
the registry but it will be more robust and capable. An integral
difference between them would be that the KB would not be transient
as the registry is. It'd be more persistent. With the combination of
dependencies and KB there is a whole lot of possibilities for Nmap
and sophistication available for scripting system. Thing that I
immediately think of is more sophisticated scripts. Other than that
more enhanced Nmap reports and delegation of data into a DBMS for
custom requirements of the user would also be possible.


* Addition of a group field for scripts

As I feel, this may or may not be as important as the above two, but
adds another grain to the usability. Currently we have 6 script
categories (+1 category as 'demo'). This notation as safe, intrusive,
discovery, malware, version and vulnerability (and more in the
future) is ideal for users who want to have control over which
type of probes are sent and which amount of noise she makes. I think
other than this classification, we could have a grouping section.
For example we could have groups like ftp, http, ssh, p2p, rpc,
windows, etc. In this way user would have more control and 
flexibility over their Nmap scans. They can select the specific 
groups they want to run. However this would not be a replication of
the behavior expected with category field, as category field can be
used to control the types of probes and noise sent.


* Swapped roles for ID field and a new 'name' field

Currently what we have is an arbitrary 'id' field which is displayed
in output. We can simply introduce a new field 'name' for that
purpose. But I think it's better to use a unique ID which will not
be change or faced with ambiguities overtime. It can be a number
or something like an alphanumeric string. Since we are not having
documentation or more info about individual scripts, in the long run
it may be possible to provide some means for that via web. Use of a
unique id will enable it among other things. For example id can be
used internally (in NSE) to handle and manage scripts, especially if
we opt for dependency/KB scenario. It may or may not be appropriate
for script writers to use script names rather than the id, based on
the practice we choose.


* Command line flag to negate scripts

I'm thinking something to do the opposite of --script here. It should
be useful to disable a specific script or a list of scripts from
execution. I found the lack of this when I was debugging an error
after I did some modification to a number of scripts. However to the
end user this would surely help in similar or different situations.


* Use of advanced Nmap techniques for more useful scripts

Currently it is advised to use scripts with a limited set of features.

For an example what I mean is most of the 31 scripts in the current
collection is just using a basic port number matching to identify a
service. While this may be an acceptable method for things like
echoTest.nse, use of this technique alone will make scripts prone to
false outcome. It is just a work of couple of keystrokes to
demonstrate how this technique can be fooled. But we all know Nmap is
better than that. Nmap can use various techniques, and fingerprint
databases to come to more accurate conclusions. So I suggest to make
use of such advanced Nmap features. If any user is not ready to send
certain types of probes, like in this case version detections scan,
she should be able to enable/disable what she desires. A config
file (which I'm going to explain in next section) and/or dependency /
KB can be used for this purpose.

This is just one example. I'm sure people will come up with more
useful things. Otherwise just using failsafe rules to write scripts
is castrating the NSE, a pure waste.


* Use of a configuration file for scripts

It'll be nice to be able to configure which scripts, categories and
which groups (or which combination of these) in a file and pass it to
Nmap. There should be a default file and it should be overridden via
command line (may be a --script-config=file). In this way users will
have a lot of flexibility and ability to plan the scans. This option
would come really handy if used efficiently. For example we could use
a dictionary file for dictionary based checks for password and mention
to use it in the config file. Also the situation I explained in the
previous section is also handled this way.


* Implement a packet forging system

This may not be directly related to NSE. The idea is to implement
a system where a user can forge custom packets. The importance is
multifold. I'm sure enthusiastic users will come up with exciting
things, but what I initially thought was, vulnerability detection,
since there's are quite a few of vulnerabilities which can be
detected (or exploited, in that matter) using a specifically
formatted packets. However after discussing this with Fyodor, I also
agreed that, may be this feature should wait till NSE gets more
stabilized. Fyodor also pointed me to a patch by Marek Majkowski on
some raw packet capability. Anyone interested should refer to
http://seclists.org/nmap-dev/2007/q1/0127.html


References:
1. NSE - http://insecure.org/nmap/nse/



Well, that's all for now. Hope it helps. As I said earlier these are
things I came across when planning scripts. Please share your ideas
about these suggestions on nmap-dev.

Cheers,
Gav


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org