Re: Suggestion: Using script output - Human-readable and machine-parseable

[David Fifield <david () bamsoftware com>]

On Thu, Apr 07, 2011 at 06:54:48AM -0500, Daniel Miller wrote:
> > For these reasons, I wonder if it would be better to create a general
> > results formatting function which takes a Lua table of results and
> > creates both XML and normal output from that?
>
> As I read your feedback, one of the more drastic changes I considered
> was changing the interface between the scripts and the Nmap program.
> Right now, scripts return a string, which Nmap has to make work in
> each output format. For this case, YAML makes sense, since it is both
> a string and a parseable representation of a data structure. Instead,
> we could have scripts return a table, which Nmap would then format
> differently for different output formats. The trouble is, no matter
> what, there has to be a linkage in the middle. Either Nmap has to
> parse YAML from the scripts, or it has to know how to handle  a Lua
> table for output.

We should have such alternate internal representations anyway. This is
what I'm thinking:

1. Scripts should still be able to return a dumb string. They must not
   all be forced to use a particular output format. XML output has to
   remain capable of expressing this.
2. Scripts can return a table of structured data instead of a string,
   along with a magic tag "yaml" that causes Nmap to serialize to YAML
   in normal output, and build a tree of elements in XML output. We
   don't embed YAML in XML so parsers don't need to parse YAML. I don't
   care if the table remains as a Lua table or is converted to another
   data structure inside Nmap.
3. It should be possible to return other such tagged data--the use case
   I'm thinking of specifically is error messages. We should have
   separate, easy-to-search-for <error> tags in XML. It should be
   possible to also return normal output in addition to an error.

The YAML part of this is not really important, if we expect people to
get their structured script output from the XML output only. In normal
output we could make it look like whatever we want. YAML looks good
though, pretty much like stdnse.format_output does. Those scripts for
which it doesn't look good can continue to return a string.

> > I'd also like to hear what people think about the option of
> > writing a function/library which takes a Lua table of results data and
> > outputs it in nice user-readable form (like we already have) and also
> > XML.

This is exactly the idea I had for this project; sorry if I wasn't
clear.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/