Re: [NSE] New class of scripts -- New Net Rules proposal

[Fyodor <fyodor () insecure org>]

On Sat, Jun 26, 2010 at 11:42:42AM +0100, Djalal Harouni wrote:
> Diffs between this proposal and the first one:

Thanks for sending the updated proposal!  The last one is good, and
this one is even better.  I do have some suggestions and notes.

In the NSE meeting, we talked about registering scripts for netrules
by adding special categories (like we do with the "version" category
for version detection scripts) or by having separate rules for the
different script scan phases (like the way we have separate host and
port script rules).  As noted in your mail, we decided to go with new
rules for net scripts.

For consistency, the plan is to change version detection scripts to
work the same way.  There will be a version rule which the version
detection scripts will have.  A description of this change (like what
arguments they should take, etc.) should probably be either added to
your writeup, or written up separately.

>   Maybe future cases:
>   * Before hostgroup NSE scan.
>   * After hostgroup NSE scan.

Maybe, but I haven't yet seen any examples where this is needed.
Particularly given that we already have a much more powerful
dependency system for controlling order during script scanning of a
hostgorup.

> o Only the pre-scan scripts have the ability to push new targets to
> Nmap.

I agree that post-scan scripts shouldn't have that ability.  But I
think hostrule and portrule scripts should have that ability unless
there is some problem with doing so.  If a zone-transfer pre-scan
netwrule script can add IPs, why shouldn't it be able to do so when it
or a similar script runs as a portrule against a nameserver discovered
by Nmap?  Or ntp-monlist when it finds new NTP peers?  Note that I
don't think these should do so by default, just that they should have
the capability to do so if the user requests it.

> o Since the scripts are identified by the new netrules then a new Nmap
> argument must be added "--script-netscan" to let NSE evaluates these
> netrules in order to run the scripts. 

I don't think we need this.  I think NSE should always evaluate the
netrules (and other rule types) for all the scripts specified by the
user.  After all, we don't have --script-portscan to enable portrule
scripts or --script-hostscan to enable host rule scripts.  The scripts
will only run if the user requests them or if they are in the default
category (and script scanning is requested) anyway.  If there is a
script which has some combination of netrules, portrules, and
hostrules, and it is desirable to control which rules are executed,
that hypothetical script can use script args.  If a certain script arg
becomes extremely popular, we can look at adding a special Nmap option
for it.  But I'd wait until we need it.


> o The netrules and action functions will receive a new net lua table.

I'm not so sure about this table.  If it is really important, maybe
host and port scan scripts should receive it too?  Just like we give
the host table to port rules and not just host rules.  Or is it really
only important for netrule scripts? Let's look at the table:

> 5) Net table:
> -------------
> Information passed to the new net scripts is in the net lua table.
>
> o net -- This table is passed as a parameter to the rule and action
> functions.
>
> o net.command_line_targets -- The targets specified on the command line 
> (including -iR and -iL).

I'm not sure I understand this, or why we need it.  So if the user
specifies "-iR 0", this table would contain "-iR 0"?  If this is
indeed important for net scripts, why isn't it important for host and
port scripts too?

> o net.excluded_targets -- The targets specified to be excluded via
> --exclude and --excludefile.

Maybe we should treat this like the new port exclusion code and export
a function for testing whether an IP is excluded?  Or maybe we should
just let Nmap skip excluded hosts even if scripts try to push them on
the scan queue.

> o net.targets -- The "effective" targets. This should be the
> net.command_line_targets - net.excluded_targets.

So if the user specified "nmap --exclude 10.0.41.7 10.0.0.0/8", what
would "net.targets" contain?  "10.0.41.7"?

> o net.interface -- like the host.interface.

If my scan is for "nmap localhost scanme.nmap.org", what is the
net.interface?  host.interface makes sense because we only have one
interface for reaching a given host.  But a scan can contain many
interfaces.  Maybe this could be an array of interfaces, but we should
make sure we have scripts which need that first.

> o net.mac_addr_src -- like the host.mac_addr_src.
> o net.bin_ip_src -- like the host.bin_ip_src.

These have the same issues.  A scan may include many target hosts
which may invove different src mac addresses and IP addresses.

> o The net target fields reflect Nmap targets. These fields can be useful
> to parse Nmap targets (used ones from the excluded ones).

I'd like to see examples of scripts where this is necessary or useful.
I'm not saying there aren't any--just that I'd like to see some to
better understand how these are used.

> o The net.targets field can be ignored by these scripts, of course there
> are other net scripts that depend on Nmap targets to work.
> e.g: script which does broadcast stuff on a network target needs this
> field which is the specified net. After that the script can parse the
> netblock etc.

Or the script could use script args to determine which nets to
broadcast on.  Because if it uses the Nmap targets, Nmap will scan
that whole range too, which might be undesired behavior.

> another e.g: use AS entries to get IPs, in this case the script can 
> ignore the cmdline targets in other words ignore this field, and use the
> --script-args for its input.

Yes, I think --script-args could be used for both examples above.

> A big point which will be discussed in the next proposal:
> If NSE host discovery scripts will expand the Nmap target list or
> narrow it down like the ping scan ?
> Please keep in mind that this point will be discussed in a next
> Nmap New Targets proposal.

We definitely need scripts which expand the Nmap target list.  I think
we should only consider adding scripts which remove IPs if we can find
important use cases for that.

> Examples why the net target fields can be useful:
>
>  $ nmap --script host-broadcast-ping --script-args \
>  "host-broad-ping.net=192.168.0.0/24" 192.168.0.0/24
>
> We can have some duplicate arguments here, but we can also specify this:
>  $ nmap --script host-broadcast-ping --script-args \
>  "host-broad-ping.net=192.168.0.0/24" 192.168.0.0/18
>
> And we can also have this:
>  $ nmap --script host-broadcast-ping --script-args \
>  "host-broad-ping.net=192.168.0.0/24" --exclude=192.168.0.10 192.168.0.0/18
>  
>  With this last example the host-broadcast-ping.nse script must not
>  count the 192.168.0.10 and ignore it, so I think that the scripts must
>  know all about Nmap targets, and this information will be in the net
>  table, which is passed to the netrules and to the action function. As 
>  said above there will be some scripts which don't use the net target
>  fields.

I don't really understand these examples.  What is wrong with just:

nmap --script host-broadcast-ping --script-args host-broad-ping.net=192.168.0.0/24"

No need for the user to specify 192.168.0.0/24 as targets too unless
said user wants to scan those IPs.  As for your last example, it seems
you could exclude the target list there too:

nmap --script host-broadcast-ping --script-args "host-broad-ping.net=192.168.0.0/24" --exclude=192.168.0.10

Your examples don't include --script-netscan, and I think that is
fine.  Presumably host-broad-ping is a netrule_pre script.

> o The netrules will run when the new --script-netscan Nmap option is
> specified.

I think they should always be evaluated when a script is specified,
just as with the other scripts.

> o The action function receives two arguments:
>  * A net table argument.
>  * Another string argument "netrule_pre" or "netrule_post", in this case 
>  if there are scripts with more than one netrule the action will know 
>  when it has been called.

If we're going to give a string for this phase, we should probably
give one for the action rules of host, port, and version scripts too.

> This idea about a new class of scripts was first brought by Ron [2].
> Thx for reading and for your comments.

Thank you!

Cheers,
Fyodor
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/