Nmap Development mailing list archives
Re: [NSE] Script Dependencies Replacement for Runlevels
From: David Fifield <david () bamsoftware com>
Date: Mon, 9 Nov 2009 16:06:18 -0700
On Sun, Nov 08, 2009 at 05:45:19PM -0500, Patrick Donnelly wrote:
Right now scripts are required to assign a runlevel for their scripts to enforce an ordered execution of a group of scripts during an NSE scan. As an example, smb-brute.nse uses a runlevel of 0.5 so it runs before other smb-* scripts. This allows the other smb-* scripts to utilize the results from the smb-brute.nse script. Unfortunately, it can be difficult to identify the dependencies between these scripts (specified loosely via runlevels) and impossible to enforce a dependency (since scripts are not aware of what other scripts are running). Enforcing a dependency means that we do not run our script if we are missing a dependency (or, we abort scanning altogether because a dependency is missing).
This is a great idea. I think this patch should go in, but first, I want to see what people think about some changes to the interface.
I have created a patch to NSE that replaces runlevels with a table of dependencies that clearly outlines what other scripts the script depends on. The table is of the form: dependences = {"script1", script2", ...}
What do you think about using "depends" instead of "dependencies"? "dependencies" is hard to type. "dependencies" doesn't correspond to anything in the current system, right? There's no way currently for one script to require the execution of another? weak_dependencies is what we emulate using runlevels. I think people will have trouble with the distinction between "strong" and "weak" dependencies. How about using a name like "run_after" for weak dependencies?
Runlevels become an internal representation of the order of scripts that are generated by the dependencies. If a dependency is not present in the current group of scripts then an error will be raised noting the missing dependency. Alternatively, you can use the new command line option --script-autoadd to automatically add dependencies to the current group of scripts (this can potentially add dangerous scripts and therefore is not the default).
It took second to figure out what --script-autoadd is doing. At first I thought it was going to modify the script files themselves and add a line with dependencies inferred from their runlevel ("automatically add dependencies to the current group of scripts"), but that's not it at all. Really, it's this: If a strong dependency is not satisfied by the scripts you list, you get an error. So, for example I made html-title depend on http-auth: ./nmap --datadir . --script=html-title scanme.nmap.org -p 80 NSE: failed to initialize the script engine: ./nse_main.lua:516: missing dependency in chain `html-title->http-auth` (use --script-autoadd to automatically add missing dependencies) Using --script-autoadd will automatically fill in any dependencies of scripts, so you won't get that error. It's like "apt-get install ..." installs all the packages that are recursive dependencies of the one you want. ./nmap --datadir . --script=html-title --script-autoadd scanme.nmap.org -p 80 -d NSE: Script dependency './scripts/http-auth.nse' in chain `html-title->http-auth` added automatically. NSE: Loaded 2 scripts for scanning. My question is, do we need support for strong dependencies? I'm assuming I'm correct in thinking that strong dependencies are a new future, and that weak dependencies are equivalent to runlevels. If there is a use case for strong dependencies I'm not against them, but I would like to avoid having a --script-autoadd option. My bias at the moment is to just have a "depends" or "run_after" variable that works like the weak_dependencies you have proposed, and not implement strong dependencies, but I'm willing to hear potential uses for strong dependencies.
We also have weak dependencies that specify scripts that the script should run after but are not required for its execution. Its form is the same as the dependencies table above. The user will still see what the current runlevel is during the scan. Additionally, they will now be aware of the number of runlevels: NSE: Script scanning 127.0.0.1. NSE: Starting runlevel 1 (of 3) scan. Initiating NSE at 17:38 Completed NSE at 17:38, 0.00s elapsed NSE: Starting runlevel 2 (of 3) scan. Initiating NSE at 17:38 Completed NSE at 17:38, 0.00s elapsed NSE: Starting runlevel 3 (of 3) scan. Initiating NSE at 17:38 Completed NSE at 17:38, 0.00s elapsed NSE: Script Scanning completed.
This is good output.
Another non-obvious benefit to explicit dependencies is we no longer have scripts running in their own runlevel needlessly (reducing our overall parallelism). Before, smb-brute would run by itself in runlevel 0.5 when it could run alongside other unrelated scripts. With respect to backwards compatibility, there is none. We ignore any runlevel specification in the script. Explicit dependencies would be required.
I can think of one thing that runlevels offer that explicit dependencies don't. Say you have a brute force script, and you want it to run after every script that can potentially find a login for you. For example, telnet-brute might want to run after http-userdir-enum because the latter can identify usernames. We could do this with runlevels by giving a the login-finding scripts a low runlevel, but with explicit dependencies every brute force script will need to know about every login script. I suspect this is not easy to solve from a user interface point of view. One idea I had was something like weak_dependencies = {"*login"} "*login" would stand for a class of scripts that would somehow signify that they satisfy it. The runlevel assigned to the brute script would have to be at least one more than the highest runlevel of any *login script. David Fifield _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- [NSE] Script Dependencies Replacement for Runlevels Patrick Donnelly (Nov 08)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 08)
- Re: [NSE] Script Dependencies Replacement for Runlevels Fyodor (Nov 09)
- Re: [NSE] Script Dependencies Replacement for Runlevels David Fifield (Nov 09)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels David Fifield (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Fyodor (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Fyodor (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Patrick Donnelly (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Ron (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Fyodor (Nov 10)
- Re: [NSE] Script Dependencies Replacement for Runlevels Patrick Donnelly (Nov 12)