Re: [NSE] http-git patches

[David Fifield <david () bamsoftware com>]

On Fri, Dec 07, 2012 at 07:31:59PM -0600, Alex Weber wrote:
> http-git seemed to have stop working at some point,

I did a bisection and found that it stopped working at r29570, the
introduction of structured output. What went wrong is that string.gsub
returns two value: the substituted string and the number of
substitutions. This was being interpreted as structured output and
unstructured output, respectively. I fixed this in r30354 by removing
the string.gsub call.

> so I took the opportunity to change the output to a table so that we
> have some semantic XML with -oX.

Thank you for doing this. We're tring to gradually move existing scripts
to structured output, and this helps.

I'd like you to consider making some changes to the structured output.
Here is what it looks like now:

$ ./nmap -p80 --script http-git javaop.com -d -oX -
<table key="206.220.193.152:80/.git/">
  <elem>Potential Git repository found at 206.220.193.152:80/.git/ (found 5 of 6 expected files)</elem>
  <elem>Source might be at https://github.com/skullspace/breachdb</elem>
  <elem key="Interesting files">Contents of '.git/COMMIT_EDITMSG' matched patterns 'passw' (case-insensitive 
match)</elem>
  <elem key="Repository description">Unnamed repository; edit this file 'description' to name the...</elem>
  <elem key="Last commit message">Brought the github repo up-to-date with my private subversio...</elem>
  <elem key="Remote">git () github com:skullspace/breachdb.git</elem>
</table>

Parts of this could be better structured. For example, the URLs: those
should be isolated in elements of their own, not appear in the middle of
free text. (In general, structured output shouldn't have free text
unless it is something copied from the service.) In normal output you
show "found 5 of 6 interesting files"; in structured output you should
list the names of those files. Strings like "Interesting files" and
"Repository description" are bad key names. Imagine that you are a
programmer processing this data--what is the most convenient form in
which it can appear? The line
        Contents of '.git/COMMIT_EDITMSG' matched patterns 'passw' (case-insensitive match)
should definitely be broken down into subelements:
        <table key="interesting-files">
                <table>
                        <elem key="filename">.get/COMMIT_EDITMSG</elem>
                        <elem key="pattern">passw</elem>
                </table>
        </table>
Strings should not be truncated with "..." in structured output; that's
only for normal screen output. Think of a programmer using this XML: you
want everything to be there.

Generally, this is how you should approach structured output. Have a
function that returns everything interesting the script can report, in a
programmer-oriented form. That means a table with nice key names, and
subtables when you need to represent a list or hierarchy. This table
will be the first return value of your script. Then, make a function
that processes the nice programmer-oriented table and transforms it into
screen output. This is where you do things like omit the names of the 5
expected files, and truncate some strings with "...". The result of the
formatting function will be the second return value of the script. See
http-auth for an example of this: the challenges table is directly
returned from an API function; it's programmer-oriented data. The result
string contains a bunch of free text and labels for screen output.

Please add an @xmloutput section in future patches to show what the
structured output looks like.

David Fifield
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/