Status Report for Djalal #16 of 16

[Djalal Harouni <tixxdz () gmail com>]

Status Report #16 of 16

Hi,

For my last status report of this GSoC 2010, I've tried to detail all my
svn commits (not easy to do).

I'll try to stay here on nmap-dev and continue to contribute, and I'll
really miss those epic meetings and especially: batbot suggestions :)


Accomplishments:

* Added code documentation to the NewTargets class.

* Adjusted new added targets size check and added a debug message to
print the number of added targets passed to Nmap and the pending ones.

* Improved nmap.add_targets and target.add() to detect more errors.

* Added NSEdoc to target.lua library and cleaned code.

* Replaced some lua_pushinteger() calls by lua_pushnumber() ones, to
handle unsigned long (not all of them on 64-bit, but this is ok) and
signed integer values (-1).

* Fixed script engine verbosity. The verbosity value is boosted when
scripts are selected by name, to fix it we had to force some parts
of the script engine to ignore this boost.

* Fixed a bug that was preventing Nmap from getting all the new added
targets.

* Updated dns-zone-transfer script to be able to add A records to Nmap
scan queue. (This is a simple hack in:
nmap-exp/djalal/nse-scripts-tests branch, and it will be improved later).

* Posted on nmap-dev the details of the adding targets feature.

* Fixed a bug that caused Nmap (nmap-add-targets branch) to segfault
when the user did not run Nmap with the --script option (this was a
null-pointer dereference).
* Fixed the check 

* Merged the NSE Script Scanning phases documentation (scripting.xml).

* Cleaned some parts of the NewTargets class code.

* Ajusted some debug output.

* Added a new stdnse library function 'get_script_args()' to parse
script arguments. The code was merged.

* Updated some code to use the get_script_args() function: nfs-ls.nse
and nfs-statfs.nse in Nmap trunk, and target.lua in nmap-add-targets
branch.

* Changed added targets saving way. We use two different data
structures:
 A queue to push new discovered targets there.
 A binary tree (std::set) to save target specifications as strings.

How targets are added:
(First note: Targets here are Nmap targets, which means all the
Nmap supported target specifications).
 * new_targets_queue (queue of new pushed targets)
 * scanned_targets_cache (binary tree of uniq targets, we use this to
  prevent infinite scanning loops when targets are added by portrule and
  hostrule scripts (to avoid adding the same target and running Nmap
  against this target and adding it again and again ...))

  NSE Part:
  ---------
  target.add(target)
        ||
        ||    //We use the internal find function of the std::set
        ||    //to check if the target string is present.
        ||
  if (!scanned_targets_cache.find(target))
        ||
  new_targets_queue.push(target)      
              //targets here are in the queue (in the pending state).

  Nmap Part:
  ----------
  In the Nmap getting targets loop.

  target = new_targets_queue.pop()
        ||
        ||    //get the target from the FIFO.
        ||
  scanned_targets_cache.insert(target)
        ||
        ||    //we should provide a range to avoid searching
        ||    //or checking again if the target is there.
        ||    //Targets here are in the scanned cache.
        ||
  host_exp_group[count++] = target
              //target is saved in Nmap internal scan queue.

 Some notes:
  The key here is to optimize the scanned_targets_cache and how
  and whene targets are saved and filtered.
  For the current moment this design works, but we should save
  targets in the scanned_targets_cache based on their IPv4/IPv6
  addresses and not strings. I'll try to post later on nmap-dev
  more details and future directions.

* Updated target.add() function to return the number of targets that
were successfully added to the new targets queue (these targets will
be later pushed onto Nmap scan queue).

* Added a new nmap.get_targets_num() function to get the number of
added targets. This function will be used internally in target.lua
library, and perhaps we could make later a small wrapper function in
target.lua which can be used by post-scanning scripts to report results
and stats.

* Had a meeting with Patrick.


Priorities:

* Review and Clean the 'add new targets' feature and merge it.
* Make dns-zone-transfer able to add new targets to Nmap in a clean way. 
* Update Nmap book (Not scripting.xml) to show the new Script Scanning
phase documentation.
* See if we can do broadcast and multicast with dns-service-discovery
script.


N.B.: I'm doing some optimizations on nmap-add-targets branch so if you
see some strange behaviour don't worry, *incremental* svn commits are
comming :)

-- 
tixxdz
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/