Re: [NSE] Script Dependencies Replacement for Runlevels

[David Fifield <david () bamsoftware com>]

On Sun, Nov 08, 2009 at 05:45:19PM -0500, Patrick Donnelly wrote:
> Right now scripts are required to assign a runlevel for their scripts
> to enforce an ordered execution of a group of scripts during an NSE
> scan. As an example, smb-brute.nse uses a runlevel of 0.5 so it runs
> before other smb-* scripts. This allows the other smb-* scripts to
> utilize the results from the smb-brute.nse script. Unfortunately, it
> can be difficult to identify the dependencies between these scripts
> (specified loosely via runlevels) and impossible to enforce a
> dependency (since scripts are not aware of what other scripts are
> running). Enforcing a dependency means that we do not run our script
> if we are missing a dependency (or, we abort scanning altogether
> because a dependency is missing).

This is a great idea. I think this patch should go in, but first, I want
to see what people think about some changes to the interface.

> I have created a patch to NSE that replaces runlevels with a table of
> dependencies that clearly outlines what other scripts the script
> depends on. The table is of the form:
>
> dependences = {"script1", script2", ...}

What do you think about using "depends" instead of "dependencies"?
"dependencies" is hard to type.

"dependencies" doesn't correspond to anything in the current system,
right? There's no way currently for one script to require the execution
of another? weak_dependencies is what we emulate using runlevels.

I think people will have trouble with the distinction between "strong"
and "weak" dependencies. How about using a name like "run_after" for
weak dependencies?

> Runlevels become an internal representation of the order of scripts
> that are generated by the dependencies. If a dependency is not present
> in the current group of scripts then an error will be raised noting
> the missing dependency. Alternatively, you can use the new command
> line option --script-autoadd to automatically add dependencies to the
> current group of scripts (this can potentially add dangerous scripts
> and therefore is not the default).

It took second to figure out what --script-autoadd is doing. At first I
thought it was going to modify the script files themselves and add a
line with dependencies inferred from their runlevel ("automatically add
dependencies to the current group of scripts"), but that's not it at
all. Really, it's this: If a strong dependency is not satisfied by the
scripts you list, you get an error. So, for example I made html-title
depend on http-auth:

./nmap --datadir . --script=html-title scanme.nmap.org -p 80
NSE: failed to initialize the script engine:
./nse_main.lua:516: missing dependency in chain `html-title->http-auth`
(use --script-autoadd to automatically add missing dependencies)

Using --script-autoadd will automatically fill in any dependencies of
scripts, so you won't get that error. It's like "apt-get install ..."
installs all the packages that are recursive dependencies of the one you
want.

./nmap --datadir . --script=html-title --script-autoadd scanme.nmap.org -p 80 -d
NSE: Script dependency './scripts/http-auth.nse' in chain `html-title->http-auth` added automatically.
NSE: Loaded 2 scripts for scanning.

My question is, do we need support for strong dependencies? I'm assuming
I'm correct in thinking that strong dependencies are a new future, and
that weak dependencies are equivalent to runlevels. If there is a use
case for strong dependencies I'm not against them, but I would like to
avoid having a --script-autoadd option.

My bias at the moment is to just have a "depends" or "run_after"
variable that works like the weak_dependencies you have proposed, and
not implement strong dependencies, but I'm willing to hear potential
uses for strong dependencies.

> We also have weak dependencies that specify scripts that the script
> should run after but are not required for its execution. Its form is
> the same as the dependencies table above.
>
> The user will still see what the current runlevel is during the scan.
> Additionally, they will now be aware of the number of runlevels:
>
> NSE: Script scanning 127.0.0.1.
> NSE: Starting runlevel 1 (of 3) scan.
> Initiating NSE at 17:38
> Completed NSE at 17:38, 0.00s elapsed
> NSE: Starting runlevel 2 (of 3) scan.
> Initiating NSE at 17:38
> Completed NSE at 17:38, 0.00s elapsed
> NSE: Starting runlevel 3 (of 3) scan.
> Initiating NSE at 17:38
> Completed NSE at 17:38, 0.00s elapsed
> NSE: Script Scanning completed.

This is good output.

> Another non-obvious benefit to explicit dependencies is we no longer
> have scripts running in their own runlevel needlessly (reducing our
> overall parallelism). Before, smb-brute would run by itself in
> runlevel 0.5 when it could run alongside other unrelated scripts.
>
> With respect to backwards compatibility, there is none. We ignore any
> runlevel specification in the script. Explicit dependencies would be
> required.

I can think of one thing that runlevels offer that explicit dependencies
don't. Say you have a brute force script, and you want it to run after
every script that can potentially find a login for you. For example,
telnet-brute might want to run after http-userdir-enum because the
latter can identify usernames. We could do this with runlevels by giving
a the login-finding scripts a low runlevel, but with explicit
dependencies every brute force script will need to know about every
login script. I suspect this is not easy to solve from a user interface
point of view. One idea I had was something like

weak_dependencies = {"*login"}

"*login" would stand for a class of scripts that would somehow signify
that they satisfy it. The runlevel assigned to the brute script would
have to be at least one more than the highest runlevel of any *login
script.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/