[NSE] Script Pre-scanning and Post-scanning announcement

[Djalal Harouni <tixxdz () gmail com>]

Hi all,

I've been working on adding support for new NSE rules, the work is based
on the prerule_postrule proposal which can be found at:
svn://svn.insecure.org/nmap-exp/djalal/prerule_postrule_proposal.txt

The idea is to add two rules:
* prerule: for Script Pre-scanning.
* postrule: for Script Post-scanning.

The New Nmap scan phases are:
* Script Pre-scanning with the prerule, this scan is performed before
All Nmap scans.
* Classic Nmap scan phases for each group, for further details:
http://nmap.org/book/nmap-phases.html
* Script Post-scanning with the postrule, this scan is performed after
all Nmap scans.

With these new phases, script writers must choose which phase and rule
to use (prerule and/or postrule), the current standard (which can be
changed) is:
"use prerule unless you have a reason to use a postrule, where the
reason would be that it relies on some data generated during the scan,
like statistics gathering scripts".


Implementation:
---------------
This new implementation introduces three new environment variables for
scripts:
* SCRIPT_PATH: the script path.
* SCRIPT_NAME: script short name without the ".nse" extension, this is a
replacement for the old "filename".
* SCRIPT_TYPE: the type of the rule that activated the script, it can
take four values:
  "prerule"   => Script Pre-scanning  (NSE_PRE_SCAN)
  "portrule"  => Script scanning      (NSE_SCAN)
  "hostrule"  => Script scanning      (NSE_SCAN)
  "postrule"  => Script Post-scanning (NSE_POST_SCAN)
So we can have a script that can have multiple rules and can run at
different NSE Scan phases, and in order to adapte the action() function
and to let it know on which Script Scan phase we are you can check the
value of the SCRIPT_TYPE environment variable.


Example of the dns-zone-transfer implementation:
(The script result is at the end of the post)
-- rules
prerule = function() return true end
portrule = shortport.portnumber(53, 'tcp')

-- the script will run on the Pre-scanning phase if the user has
-- specified all the necessary arguments.
-- for the prerule: 'dnszonetransfer.domain' and
-- 'dnszonetransfer.server' arguments.

-- host and port table are nil for Script Pre/Post-scanning
action = function(host, port)
  local dns_server, dns_port
  ...
  if nmap.registry.args['dnszonetransfer.domain'] then
        domain = nmap.registry.args['dnszonetransfer.domain']
  end

  -- Script Pre-scanning phase.
  if SCRIPT_TYPE == "prerule" then
    if not domain then
      -- ** please note here we print an error message on debug level 3
      stdnse.print_debug(3,
        "Skipping '%s' %s, 'dnszonetransfer.domain' argument is missing.",
        SCRIPT_NAME, SCRIPT_TYPE)
      return  -- ** please note here we do not return results on errors
    end
    if nmap.registry.args['dnszonetransfer.server'] then
      -- ** setup DNS server
      dns_server = nmap.registry.args['dnszonetransfer.server']
    else
      -- ** please note here we print an error message on debug level 3
      stdnse.print_debug(3,
        "Skipping '%s' %s, 'dnszonetransfer.server' argument is missing.",
        SCRIPT_NAME, SCRIPT_TYPE)
      return  -- ** please note here we do not return results on errors
    end
    -- ** setup DNS server port
    dns_port = 53
  end

  -- Script scanning phase.
  elseif SCRIPT_TYPE == "portrule" then
    ...
    -- ** use the host and the port table.
    dns_server = host.ip
    dns_port = port.number
  end

  ...
  try(socket:connect(dns_server, dns_port))
  ...
  try(socket:send(data)))
  ...
end

Note:
For prerule/postrule scripts do not return results on errors, instead
print an error message on debug level 3, thx.
You can also find Ron's California plate script with a postrule at
svn://svn.insecure.org/nmap-exp/djalal/nse-scripts-tests/


I think that the implementation is stable, to get it:
* Find attached a patch to apply against the last Nmap trunk.
* From svn:
svn co --username=guest --password="" \
svn://svn.insecure.org/nmap-exp/djalal/nse-rules/


Conclusion:
With these new rules you are not restricted to the hostrule or portrule,
and you can take advantage of the Nmap/NSE API.

Other Advantages:
* All Network I/O are in C/C++ (Nsock library) with a bind to Lua.
* NSE automatically parallelizes network operations [1].


Our next goal is to code a new Nmap/NSE API which will offer a way for
scripts to add new targets to Nmap.
BTW this is the list of Nmap/NSE prerule/postrule TODO scripts:
o AS Number to IP ranges:
  http://seclists.org/nmap-dev/2010/q2/101
o IPv6 Neighbor Discovery Protocol:
  http://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol
o DNS service discovery (Bonjour):
  http://en.wikipedia.org/wiki/Bonjour_%28software%29
o Broadcast ping (could ping broadcast address and either report
  IPs+Mac addresses that it sees, or even add them to the scan queue
  if requested).
o Netbios Name Service
o DHCP broadcast requests
o Postrules could be created which give final reports/statistics or
  other useful output.  Like a reverse-index, which shows all the open
  port numbers individually and the hosts which had that port open
  (e.g. so you can see all the ssh servers at once, etc.)
  Admittedly you can do that pretty easy with Zenmap instead.
o Your idea.

Thx for reading and feedbacks and tests are really welcome :)


Script Pre-scanning test with dns-zone-transfer:
(*Output was modified*)
$ ./nmap -d1 --datadir . --script scripts/dns-zone-transfer.nse \
--script-args="dnszonetransfer.server=ns.example.com,dnszonetransfer.domain=example.com"
...
...
NSE: Loaded 1 scripts for scanning.
NSE: Script Pre-scanning.
NSE: Starting runlevel 1 (of 1) scan.
Initiating NSE at 00:34
NSE: NSE Script Threads (1) running:
NSE: Starting dns-zone-transfer.
NSE: Finished dns-zone-transfer.
Completed NSE at 00:34, 6.49s elapsed
Pre-scan script results:
| dns-zone-transfer:  
| example.com                           SOA     dns1.icann.org 
| example.com                           NS      a.iana-servers.net
| example.com                           NS      b.iana-servers.net                     
| example.com                           A       192.0.32.10      
| ....
| www.example.example.com               CNAME  
|_example.com                           SOA     hostmaster.icann.org
NSE: Script Post-scanning.
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 6.16 seconds


[1] http://nmap.org/book/nse-parallelism.html


-- 
tixxdz

Attachment: nse_prerule_postrule.diff.tgz
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/