Re: [NSE] New class of scripts -- New Net Rules proposal

[Djalal Harouni <tixxdz () gmail com>]

NSE proposal: New net rules


Diffs between this proposal and the first one:
http://seclists.org/nmap-dev/2010/q2/952 are shown in the table section.

Reading the first proposal will help you to follow, and to make it easy
I've marked the sections that have been changed in this new proposal.
Some new ideas came from our GSoC NSE meeting, and most of the change 
is due to the use of multiple netrules (separate rules).


Table:
------
1) Introduction                   (Minor changes)
2) More NSE Rules Background      (Not changed)
3) Motivation                     (Minor Changes)
4) New NSE net scripts details    (Changed)
5) Net table                      (Changed)
6) New Rules                      (Changed)
7) The Action function            (New)
8) Simple implementation example  (Changed)
9) Consclusion                    (Not changed)


1) Introduction:
----------------
In this proposal we will try to introduce new rules which can be used
by a new class of NSE scripts. 

The current NSE rules are:
o hostrule: specifies that the script should only run once against the
host IP.
o portrule: specifies that the script should only run against the ports
list of the current target.

The limitation of these rules is that the scripts will run only against
a single host IP (hostrule or portrule), and this host is provided by
Nmap which has already done a bunch of operations to get this information, 
so as an example, we can't run scripts against other targets which can be
networks or domain names that do not depend on Nmap previous operations.
* Network targets can be useful for host discovery and broadcast operations.
* Domain name targets can be useful to do DNS brute force operations.
Another limitation is that since NSE scripts depend on host and port
tables information they must run after Nmap scan phases.

A simple example:
* Script: dns-zone-transfer.nse
* Portrule: portrule = shortport.portnumber(53, 'tcp')
Modify the dns-zone-transfer.nse script and add another rule to let the
script run against the domain name to discover new targets. The current
script will only run when Nmap finds a DNS server, so with a new added
rule that script will run directly and does not depend on open ports and
can find new subdomain targets for Nmap, in other words specify a domain
name as a target and with the use of the results of this script, Nmap
will scan all the newly discovered subdomains and hostnames.


Please note that this is a first proposal about new rules. There will 
be other proposals to complete the New scripts class definition.


2) More NSE Rules Background:
-----------------------------
At present there are only two types of scripts, host and port classes
scripts. Each script has its own rule and action.

o The action function is the heart of an NSE script [1].
o The rule is the Lua function which will be evaluated by NSE to true or
  false.

The script's action will only be executed if its rule evalutes to true.

NSE Rules:
o Host rule:
"Specifies that the script should be run only once against a target
IP and only if the given conditions are met." [1]
The Host rule accepts a host table which contains the IP or Hostname of
the target as an argument.

o Port rule:
"Governs which ports of a target the scripts may run against" [1].
The Port rule accepts both host and port tables as arguments or any TCP
or UDP port in the open, open|filtered, or unfiltered port states [1].


3) Motivation:
--------------
1. Network operations:
We can need scripts that perform different network operations in order
to discover more hosts. In general these scripts will run once per scan
and can feed Nmap with new targets, but we can have other use cases.

 o Use broadcast and multicast protocols as suggested in this thread
   http://seclists.org/nmap-dev/2010/q1/883

 o We can also use AS entries to get IPs as suggested in this thread
   http://seclists.org/nmap-dev/2010/q2/101

 o Perform DNS discovery: brute force, zone transfert etc.

 o Perform Whois requests.

 o Any other operation which can take a network or a simple input that
 can be used to produce a list of Nmap targets.

 o Scripts that run after Nmap scans to collect/report results.


4) New NSE net scripts details:
-------------------------------
o The new scripts can run at different times:
  * Before any scanning: pre-scan scripts.
  * After Nmap scans: post-scan scripts.

  Maybe future cases:
  * Before hostgroup NSE scan.
  * After hostgroup NSE scan.


o Pre-scan scripts can be used for host discovery like DNS scripts. We
can also have other cases of use like scripts which simply want to use
Nmap/NSE/Lua API to do cool tricks.
e.g: Ron's California Vanity License plate script :-)
http://www.skullsecurity.org/blog/?p=723

o Post-scan scripts: can be used to report data.
e.g: A script which collects all the whois hostrule script contact 
details results and report the unique ones.

o Only the pre-scan scripts have the ability to push new targets to
Nmap.

o Since scripts can run at different times, then we need different rules.
 * netrule_pre -- this rule will be used for pre-scan scripts.
 * netrule_post -- this rule will be used for post-scan scripts.

o The new scripts will be identified by the presence of these netrules.

o Since the scripts are identified by the new netrules then a new Nmap
argument must be added "--script-netscan" to let NSE evaluates these
netrules in order to run the scripts. 

o The netrules and action functions will receive a new net lua table.

o Each script can have its own arguments like any other script.
  e.g: netscript.arg1=x, netscript.arg2=x ...


A big point which will be discussed in the next proposal:
If NSE host discovery scripts will expand the Nmap target list or
narrow it down like the ping scan ?
Please keep in mind that this point will be discussed in a next
Nmap New Targets proposal.


5) Net table:
-------------
Information passed to the new net scripts is in the net lua table.

o net -- This table is passed as a parameter to the rule and action
functions.

o net.command_line_targets -- The targets specified on the command line 
(including -iR and -iL).

o net.excluded_targets -- The targets specified to be excluded via
--exclude and --excludefile.

o net.targets -- The "effective" targets. This should be the
net.command_line_targets - net.excluded_targets.

o net.interface -- like the host.interface.

o net.mac_addr_src -- like the host.mac_addr_src.

o net.bin_ip_src -- like the host.bin_ip_src.


Additions:
o The net target fields reflect Nmap targets. These fields can be useful
to parse Nmap targets (used ones from the excluded ones).

o The net.targets field can be ignored by these scripts, of course there
are other net scripts that depend on Nmap targets to work.
e.g: script which does broadcast stuff on a network target needs this
field which is the specified net. After that the script can parse the
netblock etc.
another e.g: use AS entries to get IPs, in this case the script can 
ignore the cmdline targets in other words ignore this field, and use the
--script-args for its input.


Examples why the net target fields can be useful:

 $ nmap --script host-broadcast-ping --script-args \
 "host-broad-ping.net=192.168.0.0/24" 192.168.0.0/24

We can have some duplicate arguments here, but we can also specify this:
 $ nmap --script host-broadcast-ping --script-args \
 "host-broad-ping.net=192.168.0.0/24" 192.168.0.0/18

And we can also have this:
 $ nmap --script host-broadcast-ping --script-args \
 "host-broad-ping.net=192.168.0.0/24" --exclude=192.168.0.10 192.168.0.0/18
 
 With this last example the host-broadcast-ping.nse script must not
 count the 192.168.0.10 and ignore it, so I think that the scripts must
 know all about Nmap targets, and this information will be in the net
 table, which is passed to the netrules and to the action function. As 
 said above there will be some scripts which don't use the net target
 fields.


6) New Rules:
-------------
Since we can have different scripts which can run at different times or
multiple times as stated in the [New NSE net scripts details] section,
the imprortant question will be: When should the scripts run ?
For this we have proposed:

o Different netrule funtions:
 * netrule_pre -- this rule will be used for pre-scan scripts.
 * netrule_post -- this rule will be used for post-scan scripts.

o The netrules will run when the new --script-netscan Nmap option is
specified.

o The netrules will facilitate the addition of the new host discovery and
scanning scripts.

o The netrules will take a net table as an argument.

o The netrules in most cases will evaluate to true.


7) The Action function:
-----------------------
o Since new NSE scripts can run at different times, the action function
must know when it has been called, because some scripts can have
different behaviours especially when they have multiple netrules.
e.g: A whois script which have a netrule_pre and a netrule_post rules
will run at the beginning and sets a registry value to tell the whois
hostrule script to save the results onto the registry, after that the 
first whois script will run at the end (netrule_post) and collects/report 
the final results (e.g: unique contact details). Having two different
scripts, a pre-scan one and a post-scan script to do this seems to me  
rediculous.

o The action function receives two arguments:
 * A net table argument.
 * Another string argument "netrule_pre" or "netrule_post", in this case 
 if there are scripts with more than one netrule the action will know 
 when it has been called.

o This feature of a script which can have multiple netrules must not be
abused, if the action code differes a lot from when the script is in the
pre-scan mode and in the post-scan mode, then the best solution is to
have two different scripts each one with a netrule.


8) Simple implementation example:
---------------------------------

netrule_pre = shortport.netrule_pre_check("netrule_pre")

action = function(net, netrule)

        stdnse.print_debug(netrule)
        -- prints 'netrule_pre' or 'netrule_post'

        -- command line targets
        for _,v in ipairs(net.command_line_targets) do
                stdnse.print_debug("%s", v)
        end

        -- excluded targets
        for _,v in ipairs(net.excluded_targets) do
                stdnse.print_debug("%s", v)
        end

        -- effective targets
        for _,v in ipairs(net.targets) do
                stdnse.print_debug("%s", v)
        end

        stdnse.print_debug("IP Source: %s", packet.toip(host.bin_ip_src))

        return
end


9) Conclusion:
--------------
The addition of these new netrules will let the scripts take
different targets not only Nmap IP hosts or port lists, and will offer
them a way to run before or after Nmap scan phases.

This idea about a new class of scripts was first brought by Ron [2].
Thx for reading and for your comments.


[1] http://nmap.org/book/nse-script-format.html
[2] http://seclists.org/nmap-dev/2010/q1/883 


-- 
tixxdz
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/