Re: Suggestion: Using script output - Human-readable and machine-parseable

[David Fifield <david () bamsoftware com>]

On Thu, Mar 31, 2011 at 03:46:41PM -0500, Daniel Miller wrote:
> Hey list,
>
> From the Nmap TODO:
> > o Nmap should have a better way to handle XML script output.
> >  o We currently just stick the current script output text into an XML tag.
> May I make a suggestion? If we format script output as YAML  
> (http://www.yaml.org/), it will be machine-readable, while preserving  
> the human-readable nature. There are a couple of YAML bindings for Lua:  
> libyaml (http://yaml.luaforge.net/) and libsyck  
> (https://github.com/indeyets/syck/tree/master/ext/lua)
>
> The quickest way to support this straight off would be to modify  
> stdnse.format_output to output YAML. The NSE runtime could quote  
> non-YAML return values as block-literal scalars, until such time as they  
> are rewritten to return more useful formatted output. A downside would  
> be that output would take a few more lines than before, though this  
> could be overcome by using "flow styles," of which JSON is a proper 
> subset.
>
> Benefits of doing this are:
> * extensibility without requiring modifications to Nmap XML schema
> * Human-readable nature of YAML means a single output format for human  
> and machine consumers
> * existing structured-output using stdnse.format_output means very few  
> changes for script authors
> * YAML bindings available for Lua, Python, Perl, Ruby, C, and many other  
> languages.
>
> I was going to submit this idea with a patch, but I realized I don't  
> have the first clue as to how to integrate a new Lua library binding  
> into the Nmap source. Instead, here's some examples of YAML-formatted  
> script output.

How about this: Write a new stdnse.yaml function with an interface
similar to that of stdnse.format_output. Then we can switch scripts one
at a time instead of all at once.

What I am thinking is, that stdnse.yaml can somehow tag the output as
being structured data and not just plain text. Then we can modify the
XML output to look for this tag and emit an element hierarchy instead of
(or in addition to) a chunk of text. But that's a second step; the first
is just getting YAML on the screen.

Take a look at json.generate, which does something similar.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/