Re: Another look at Nmap XML

[David Fifield <david () bamsoftware com>]

On Mon, Jun 28, 2010 at 03:06:08AM +0700, sitney () gmail com wrote:
> XML output should be the most robust output format. Ideally, XML
> output contains all of the information pertaining to a scan that one
> would need. However, there are a few key items that I have observed
> are not listed in XML output: 
>
> * Host Status - i.e. If a host is down (or skipped), the record is not
> created in XML. Standard output shows this detail however. If the
> argument is that this will waste storage space with all the extra
> text, then enable the verbose -v (-vv) switch to print these records
> to -oX. This is a vital detail that really should be in -oX. I have
> numerous use cases for why I say this.

Nmap shows down hosts with -v since 5.30BETA1.

> * Hostname PTR resolution status - If a rDNS record cannot be
> resolved, let -oX know about that. This omission causes Nmap::Parser
> to choke and spit out an error (eest on freenode recently submitted a
> patch to correct this after we identified the issue). Also, the
> hostnames  order when displayed in HTML (user and PTR) are often
> reversed inconsistently. May require a minor XSLT tweak.

The fact that rDNS resolution failed is arguably already represented by
the absence of a hostname element. Scanning a target with no rDNS
produces this:

<host starttime="1277780978" endtime="1277780981"><status state="up" reason="user-set"/>
<address addr="1.2.3.4" addrtype="ipv4"/>
<hostnames>
</hostnames>
<ports><extraports state="filtered" count="100">
<extrareasons reason="no-responses" count="100"/>
</extraports>
</ports>
</host>

Would you prefer that this be shown in another way? Can you provide an
example of what the XML would look like?

> * # of IPs - If a hostname has multiple IPs, standard output will
> indicate how many there were and Nmap proceeds scanning the first one
> found. This number is omitted from -oX. Ideally, all the IPs for a
> hostname will be listed in -oX, but at the very least, please include
> the # of IPs found. This is often an indicator of a load balancer
> which is a useful piece of information.

You are right that this isn't shown in the XML. What do you think the
output should look like? Part of me just wants to stuff all the
addresses into the host element:

<host starttime="1277781472" endtime="1277781474"><status state="up" reason="syn-ack"/>
<address addr="74.125.19.147" addrtype="ipv4"/>
<address addr="74.125.19.104" addrtype="ipv4"/>
<address addr="74.125.19.103" addrtype="ipv4"/>
<address addr="74.125.19.99" addrtype="ipv4"/>
<hostnames>
<hostname name="google.com" type="user"/>
<hostname name="nuq04s01-in-f147.1e100.net" type="PTR"/>
</hostnames>

That's not so good once we add the planned option to scan all the
addresses associated with a name--would we repeat all the addresses in
each host element?

> Also, as an XML newbie, it took me a long time to find a very helpful
> piece of information: If you want to look at your XML output in your
> browser, you have to transform it to HTML with xsltproc first like
> this:
>
> $ xsltproc filename.xml > filename.html

Did you find the instructions here?

http://nmap.org/book/output-formats-output-to-html.html

There are also command lines for some other XSLT processors. It used to
be possible to open the XML file in a web browser and use its built-in
XSLT processor, but that doesn't work as well these days as browsers
enforce restrictions on where XSLT transforms can be loaded from.

> At the bottom of the html output screen, offer a "click to download in
> .csv format" button. This will get most users outputting to XML
> knowing that they can easily download a greppable file from their
> browser, and some will undoubtedly see the power of Nmap XML output
> and find ways to stick with it.

Can you explain more about how this will work? Is the ".csv format" the
same as grepable output?

> I am not an XSLT expert by any stretch, but I may take this challenge
> up myself and see if I can't update the stylesheet with this
> functionality and contribute to Nmap in some positive way(s).

I hope you decide to give this a try. A good easy thing to start with
would be consistent ordering of user/PTR names. "user" names didn't
exist when the XSLT was last updated, and it appears to be just sorting
the names alphabetically.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/