Re: NSE Online Documentation - some minor issues

[jah <jah () zadkiel plus com>]

Hi!

I wrestled LuaLogging into submission and got it to log messages from
luadoc to the console.  I don't know whether it's my lua installation
that prevented it from working out of the box, but if you have
LuaLogging installed and it doesn't work for you either then you might
be interested in the attached util.lua.patch for
nsedoc/src/luadoc/util.lua - it's nothing more than a dirty hack, but
having this info is quite useful as I'll demonstrate.

  INFO processing file `scripts/SSLv2-support.nse'
  ERROR undefined handler for tag `ouput'

  INFO processing file `nselib/datafiles.lua'
  ERROR undefined handler for tag `type'

  INFO processing file `nselib/dns.lua'
  ERROR undefined handler for tag `result'
  ERROR undefined handler for tag `result'

  INFO processing file `nselib/netbios.lua'
  ERROR undefined handler for tag `returns'

  INFO processing file `nselib/snmp.lua'
  ERROR undefined handler for tag `result'

Those are minor typos in tag names for which I've already committed fixes.

  INFO generating file `docs/modules/nmap.html'
  ERROR unresolved reference to symbol `nmap.new_dnet'
  ERROR unresolved reference to symbol `socket:pcap_register'
  ERROR unresolved reference to symbol `socket:pcap_open'

The above stem from the inability to generate href attributes for links
in @see tags and happen because the code compares the the function name
(as found in the document) to the function name (as found in the @see
tag), but with the module name (or object) stripped.
e.g. if "socket:pcap_register" == "pcap_register" then ..
which clearly won't match and the href attribute won't be created (and
the html anchor will point to the current document rather than to a
specific function).

There's a couple of approaches we could take to solve this.  The first
would be to correct the affected luadoc file such that function names
aren't prepended with objects or module names.  This is how it's done in
other libraries and scripts, but would require an example in a usage tag
to make it clear how the function should be called e.g
   --- description
   -- @usage socket = nmap.new_socket()
   function new_socket()

This function could be referenced from within the same document with or
without the module name: @see new_socket (naturally a reference from
anywhere else would require the module name).
Functions listed on the doc page would be shown without the module name
(which again, is how it's done for other libraries) and this I think is
the only potentially contentious point (although luadoc design favours
this).

Another approach might be to modify nsedoc/src/luadoc/doclet/html.lua to
perform an additional test which compares the function name found in the
document with the module names prepended to the function names. Some
other trickery would be required for functions written like
socket:close(), but it would be doable and would allow the list of
functions to be displayed in this way without breaking links from @see tags.
Personally, I think this is a less desirable way to go, but what do
/you/ think?

I've attached nmap.luadoc.patch which handles the former of these
approaches, but some of the usage examples (particularly those for
ethernet functions) require better examples.

And similarly...

  INFO generating file `docs/modules/stdnse.html'
  ERROR unresolved reference to symbol `string.format'

There is no string.luadoc so obviously a link to it isn't possible.  The
comments in stdnse reference "Lua's string.format" and so I've removed
the @see tag as part of the previously mentioned commit.

So LuaLogging is useful for finding this stuff and it also lists a load
of warnings which I haven't looked into yet:

INFO processing file `nselib/dns.lua'
WARN documenting undefined parameter `dec'
WARN documenting undefined parameter `retAll'
WARN documenting undefined parameter `dec'
WARN documenting undefined parameter `retAll'
WARN documenting undefined parameter `dec'
WARN documenting undefined parameter `retAll'
WARN documenting undefined parameter `dec'
WARN documenting undefined parameter `retAll'
WARN documenting undefined parameter `dec'
WARN documenting undefined parameter `retAll'
WARN documenting undefined parameter `entry'
WARN documenting undefined parameter `entry'
WARN documenting undefined parameter `entry'
WARN documenting undefined parameter `entry'
WARN documenting undefined parameter `data'
WARN documenting undefined parameter `pos'
WARN documenting undefined parameter `entry'
WARN documenting undefined parameter `data'
WARN documenting undefined parameter `pos'
WARN documenting undefined parameter `Flags'

INFO processing file `scripts/whois.nse'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'
WARN documenting `field' for block that is not a `table'

INFO processing file `scripts/zoneTrans.nse'
WARN parameter `name' not defined [[data]]: skipping
WARN parameter `name' not defined [[offset]]: skipping
WARN parameter `name' not defined [[ttype]]: skipping
WARN parameter `name' not defined [[table]]: skipping
WARN parameter `name' not defined [[data]]: skipping
WARN parameter `name' not defined [[offset]]: skipping

INFO processing file `nselib/match.lua'
WARN documenting undefined parameter `The'

INFO processing file `nselib/nmap.luadoc'
WARN documenting undefined parameter `protocol'
WARN parameter `name' not defined [[packet]]: skipping

INFO processing file `nselib/smb.lua'
WARN documenting undefined parameter `list'
WARN documenting undefined parameter `return'
WARN documenting undefined parameter `return'
WARN documenting undefined parameter `data'

INFO processing file `nselib/snmp.lua'
WARN parameter `name' not defined [[enterpriseOid]]: skipping
WARN parameter `name' not defined [[agentIp]]: skipping
WARN parameter `name' not defined [[genTrap]]: skipping
WARN parameter `name' not defined [[specTrap]]: skipping
WARN parameter `name' not defined [[timeStamp]]: skipping

INFO processing file `nselib/ssh2.lua'
WARN documenting undefined parameter `received'

INFO processing file `nselib/strbuf.lua'
WARN documenting undefined parameter `sbuf'
WARN documenting undefined parameter `delimiter'

INFO processing file `nselib/url.lua'
WARN documenting undefined parameter `s'
WARN documenting undefined parameter `corresponding'
WARN documenting undefined parameter `dictionary'

Regards,

jah

--- nmap.luadoc.orig    2008-10-18 21:40:32.171875000 +0100
+++ nmap.luadoc 2008-10-18 21:40:15.015625000 +0100
@@ -11,21 +11,22 @@
 -- \n\n
 -- The debugging level can be set with the -d option.
 -- @return The debugging level.
--- @usage if nmap.debugging() > 0 then ... end
-function nmap.debugging()
+-- @usage  if nmap.debugging() > 0 then ... end
+function debugging()
 
 --- Determines whether Nmap was compiled with SSL support.
 -- \n\n
 -- This can be used to avoid sending SSL probes when SSL is not available. 
 -- @return true if Nmap was compiled with SSL support, false otherwise.
-function nmap.have_ssl()
+-- @usage  if nmap.have_ssl() then ... end
+function have_ssl()
 
 --- Returns the verbosity level as a non-negative integer.
 -- \n\n
 -- The verbosity level can be set with the -v option.
 -- @return The verbosity level.
--- @usage if nmap.verbosity() > 0 then ... end
-function nmap.verbosity()
+-- @usage  if nmap.verbosity() > 0 then ... end
+function verbosity()
 
 --- Searches for the specified file and returns a string containing its path if
 -- it is found and readable (to the process).
@@ -36,13 +37,15 @@
 -- "/usr/local/share/nmap/nmap-rpc".
 -- @param filename Filename to search for.
 -- @return String representing the full path to the file or nil.
-function nmap.fetchfile(filename)
+-- @usage  local filepath = nmap.fetchfile(filename)
+function fetchfile(filename)
 
 --- Returns the timing level as a non-negative integer. Possible return values
 -- vary from 0 to 5, corresponding to the six built-in Nmap timing templates.
 -- The timing level can be set with the -T option.
 -- @return The timing level.
-function nmap.timing_level()
+-- @usage  local foo = nmap.timing_level()
+function timing_level()
 
 --- Gets a port table for a port on a given host.
 -- \n\n
@@ -63,7 +66,7 @@
 -- @param protocol Protocol string ("tcp" or "udp")
 -- @return A new port table holding the status and information for the port.
 -- @usage p = nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"})
-function nmap.get_port_state(host, port)
+function get_port_state(host, port)
 
 --- Sets the state of a port on a given host.
 -- \n\n
@@ -76,7 +79,8 @@
 -- @param host Host table, containing an "ip" field.
 -- @param port Port table, containing "number" and "protocol" fields.
 -- @param state Port state, like "open" or "closed".
-function nmap.set_port_state(host, port, state)
+-- @usage nmap.set_port_state(host, port, "open")
+function set_port_state(host, port, state)
 
 --- Sets version information on a port.
 -- \n\n
@@ -103,12 +107,14 @@
 -- any additional version information fields.
 -- @param probestate The state of the probe: "hardmatched", "softmatched",
 -- "nomatch", "tcpwrapped", or "incomplete".
-function nmap.set_port_version(host, port, probestate)
+-- @usage nmap.set_port_version(host, port, "hardmatched")
+function set_port_version(host, port, probestate)
 
 --- Returns the current date and time in milliseconds.
 -- @return The number of milliseconds since the epoch (on most systems this is
 -- 01/01/1970).
-function nmap.clock_ms()
+-- @usage  local now = nmap.clock_ms()
+function clock_ms()
 
 --- Gets the link-level hardware type of an interface.
 -- \n\n
@@ -117,7 +123,8 @@
 -- "ethernet", "loopback", "p2p", or nil if none of the other types apply.
 -- @param interface_name The name of the interface.
 -- @return "ethernet", "loopback", "p2p", or nil.
-function nmap.get_interface_link(interface_name)
+-- @usage local iface = nmap.get_interface_link("eth0")
+function get_interface_link(interface_name)
 
 --- Create a mutex on an object.
 -- \n\n
@@ -149,7 +156,7 @@
 --   mutex "done";\n
 --   return script_output;\n
 -- end
-function nmap.mutex(object)
+function mutex(object)
 
 --- Creates a new exception handler.
 -- \n\n
@@ -174,7 +181,7 @@
 -- try(socket:connect(host.ip, port.number))\n
 -- result = try(socket:receive_lines(1))\n
 -- try(socket:send(result))
-function nmap.new_try(handler)
+function new_try(handler)
 
 --- Returns a new NSE socket object.
 -- \n\n
@@ -192,8 +199,9 @@
 -- connect-style sending and receiving over TCP and UDP (and SSL), as well as
 -- raw socket receiving.
 -- @return A new NSE socket.
--- @see nmap.new_dnet
-function nmap.new_socket()
+-- @see    new_dnet
+-- @usage  local socket = nmap.new_socket()
+function new_socket()
 
 --- Establishes a connection.
 -- \n\n
@@ -227,7 +235,9 @@
 -- @param protocol "tcp", "udp", or "ssl" (default "tcp").
 -- @return Status (true or false).
 -- @return Error code (if status is false).
-function socket:connect(hostid, port, protocol)
+-- @see    new_socket
+-- @usage  local status, err = socket:connect(host.ip, port, "udp")
+function connect(hostid, port, protocol)
 
 --- Sends data on an open socket.
 -- \n\n
@@ -246,7 +256,9 @@
 -- @param data The data to send.
 -- @return Status (true or false).
 -- @return Error code (if status is false).
-function socket:send(data)
+-- @see    new_socket
+-- @usage  local status, err = socket:send(data)
+function send(data)
 
 --- Receives data from an open socket.
 -- \n\n
@@ -259,7 +271,9 @@
 -- send operation.
 -- @return Status (true or false).
 -- @return Data (if status is true) or error string (if status is false).
-function socket:receive()
+-- @see    new_socket
+-- @usage  local status, response = socket:receive()
+function receive()
 
 --- Receives lines from an open connection.
 -- \n\n
@@ -275,7 +289,9 @@
 -- @param n Minimum number of lines to read.
 -- @return Status (true or false).
 -- @return Data (if status is true) or error string (if status is false).
-function socket:receive_lines(n)
+-- @see    new_socket
+-- @usage  local status, response = socket:receive_lines(1)
+function receive_lines(n)
 
 --- Receives bytes from an open connection.
 -- \n\n
@@ -290,7 +306,9 @@
 -- @param n Minimum number of bytes to read.
 -- @return Status (true or false).
 -- @return Data (if status is true) or error string (if status is false).
-function socket:receive_bytes(n)
+-- @see    new_socket
+-- @usage  local status, response = socket:receive_bytes(1)
+function receive_bytes(n)
 
 --- Reads from a socket using a buffer and an arbitrary delimiter.
 -- \n\n
@@ -331,7 +349,9 @@
 -- data.
 -- @return Status (true or false).
 -- @return Data (if status is true) or error string (if status is false).
-function socket:receive_buf(delimiter, keeppattern)
+-- @see    new_socket
+-- @usage  status, response = socket:receive_buf("\r?|n", false)
+function receive_buf(delimiter, keeppattern)
 
 --- Closes an open connection.
 -- \n\n
@@ -346,7 +366,9 @@
 -- difficult to predict, it is considered good practice to close opened sockets.
 -- @return Status (true or false).
 -- @return Error code (if status is false).
-function socket:close()
+-- @see    new_socket
+-- @usage  socket:close()
+function close()
 
 --- Gets information about a socket.
 -- \n\n
@@ -363,7 +385,9 @@
 -- @return Local port number (if status is true).
 -- @return Remote IP address (if status is true).
 -- @return Remote port number (if status is true).
-function socket:get_info()
+-- @see    new_socket
+-- @usage  local status, lhost, lport, rhost, rport = socket:get_info()
+function get_info()
 
 --- Sets a timeout for socket input and output operations.
 -- \n\n
@@ -371,7 +395,9 @@
 -- return. The default value is 30,000 (30 seconds). The lowest allowed value is
 -- 10 ms, since this is the granularity of NSE network I/O.
 -- @param t Timeout in milliseconds.
-function socket:set_timeout(t)
+-- @see   new_socket
+-- @usage socket:set_timeout(10000)
+function set_timeout(t)
 
 --- Opens a socket for raw packet capture.
 -- \n\n
@@ -389,8 +415,9 @@
 -- @param test_function Callback function used to compute the packet hash.
 -- @param bpf A string describing a Berkeley packet filter expression (like
 -- those provided to tcpdump).
--- @see socket:pcap_register
-function socket:pcap_open(device, snaplen, promisc, test_function, bpf)
+-- @see   new_socket, pcap_register
+-- @usage socket:pcap_open(device, snaplen, promisc, test_function, bpf)
+function pcap_open(device, snaplen, promisc, test_function, bpf)
 
 --- Starts listening for incoming packages.
 -- \n\n
@@ -399,8 +426,9 @@
 -- to receive all packets, just provide the empty string (""). There has to be a
 -- call to pcap_register() before a call to pcap_receive().
 -- @param packet_hash A binary string that is compared against packet hashes.
--- @see socket:pcap_open
-function socket:pcap_register(packet_hash)
+-- @see   new_socket, pcap_open
+-- @usage socket:pcap_register(packet_hash)
+function pcap_register(packet_hash)
 
 --- Receives a captured packet.
 -- \n\n
@@ -413,20 +441,27 @@
 -- parameter is smaller than the total packet length).
 -- @return Data from the second OSI layer (e.g. ethernet headers).
 -- @return Data from the third OSI layer (e.g. IPv4 headers).
-function socket:pcap_receive()
+-- @see    new_socket, pcap_register, pcap_open
+-- @usage  status, plen, l2_data, l3_data = socket:pcap_receive()
+function pcap_receive()
 
 --- Closes the pcap device.
-function socket:pcap_close()
+-- @see   new_socket, pcap_open
+-- @usage socket:pcap_close()
+function pcap_close()
 
 --- Creates a new dnet object, used to send raw packets.
-function nmap.new_dnet()
+-- @usage local dnet = nmap.new_dnet()
+function new_dnet()
 
 --- Opens an ethernet interface for raw packet sending.
 -- \n\n
 -- An error ("device is not valid ethernet interface") is thrown in case the
 -- provided argument is not valid.
 -- @param interface_name The dnet-style name of the interface to open.
-function dnet:ethernet_open(interface_name)
+-- @see   new_dnet
+-- @usage dnet:ethernet_open("eth0")
+function ethernet_open(interface_name)
 
 --- Sends a raw ethernet frame.
 -- \n\n
@@ -435,10 +470,14 @@
 -- ethernet header. If there was no previous valid call to ethernet_open() an
 -- error is thrown ("dnet is not valid opened ethernet interface").
 -- @param packet
-function dnet:ethernet_send(packet)
+-- @see   new_dnet
+-- @usage dnet:ethernet_send(packet)
+function ethernet_send(packet)
 
 --- Closes an ethernet interface.
 -- \n\n
 -- An error ("device is not valid ethernet interface") is thrown in case the
 -- provided argument is not valid.
-function dnet:ethernet_close()
+-- @see   new_dnet, ethernet_open
+-- @usage dnet:ethernet_close()
+function ethernet_close()

--- util.lua.orig       2008-10-18 15:09:04.406250000 +0100
+++ util.lua    2008-10-18 15:05:06.265625000 +0100
@@ -3,6 +3,9 @@
 -- @release $Id: util.lua,v 1.16 2008/02/17 06:42:51 jasonsantos Exp $
 -------------------------------------------------------------------------------
 
+local require = require
+local logging = require "logging"
+local os = require "os"
 local lfs = require "lfs"
 local type, table, string, io, assert, tostring, setmetatable, pcall = type, table, string, io, assert, tostring, 
setmetatable, pcall
 
@@ -165,37 +168,39 @@
 -- @return logger object that will implement log methods
 
 function loadlogengine(options)
-       local logenabled = pcall(function()
-               require "logging"
-               require "logging.console"
-       end)
-       
-       local logging = logenabled and logging
-       
-       if logenabled then
-               if options.filelog then
-                       logger = logging.file("luadoc.log") -- use this to get a file log
-               else
-                       logger = logging.console("[%level] %message\n")
-               end
-       
-               if options.verbose then
-                       logger:setLevel(logging.INFO)
-               else
-                       logger:setLevel(logging.WARN)
-               end
-               
-       else
-               noop = {__index=function(...)
-                       return function(...)
-                               -- noop
-                       end
-               end}
-               
-               logger = {} 
-               setmetatable(logger, noop)
-       end
-       
-       return logger
+  if true then
+    if options.filelog then
+      logger = logging.file("luadoc.log") -- use this to get a file log
+    else
+      logger = logging.console()
+    end
+
+    if options.verbose then
+      logger:setLevel(logging.INFO)
+    else
+      logger:setLevel(logging.WARN)
+    end
+
+  else
+    noop = {__index=function(...)
+      return function(...)
+        -- noop
+      end
+    end}
+
+    logger = {}
+    setmetatable(logger, noop)
+  end
+
+  return logger
+end
+
+function logging.console(logPattern)
+
+    return logging.new(  function(self, level, message)
+                            io.stdout:write(logging.prepareLogMsg(logPattern, os.date(), level, message))
+                            return true
+                        end
+                      )
 end
 


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org