Re: Suggestion: Using script output - Human-readable and machine-parseable

[Fyodor <fyodor () insecure org>]

On Tue, Apr 05, 2011 at 07:35:29AM -0500, Daniel Miller wrote:
> I'm attaching a patch that allows for formatting output as YAML.

Thanks Daniel, this is a very interesting idea (and implementation)!
For comparison's sake, another option we could consider is formatting
script results in XML (for structured XML output) and a separate
human-readable format (as we do now) for normal output.

The big advantage of YAML is that it is one format which can serve
both purposes.  This would make it easier for script writers to use,
and would also allow folks to more easily parse the script output from
Nmap's normal-format output.

But my worry is that this single format might be worse for both
purposes.  The XML output (which is the version meant for parsing)
will be harder to parse because programs doing so will now need a YAML
parser in addition to an XML parser.  We would have to add that to
Zenmap and Ndiff if we want to make use of the results, and so would
3rd party programs which use Nmap.

Meanwhile, the user-visible output is likely to suffer to because it
would be constrained to the YAML format.  So, for example, we end up
with ugly labels for a bunch of elements even when it is obvious to
the user what the elements represent.

For these reasons, I wonder if it would be better to create a general
results formatting function which takes a Lua table of results and
creates both XML and normal output from that?  For normal output, it
would do things like add the indenting and might ignore the labels in
some or all cases.  For XML, it would use the labels as attributes in
XML tags providing the data.  There would probably be an XML element
containing the user-formatted string results too (as there is now).

All of Nmap's output could potentially be formatted in YAML, but our
approach has instead been to make the normal output as human-readable
as possible and to provide parallel results in XML for parsing.

I haven't thought about this issue as much as others have, and my YAML
knowledge is basically limited to Daniel's previous posts on the topic
and the example script output in his emails.  So I'm not against the
YAML approach if it proves to be the best.  I just want to see what
ideas people have about the potential problem of YAML being harder to
read in human-readable output and harder to parse when embedded in
XML.  I'd also like to hear what people think about the option of
writing a function/library which takes a Lua table of results data and
outputs it in nice user-readable form (like we already have) and also
XML.

Cheers,
Fyodor
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/