Another look at Nmap XML

[sitney () gmail com]

Greetings,

I wanted to chime in on the XML discussion that has come up at times.  My observations here should be viewed not as 
expert opinion, but as an Nmap user struggling to move away from standard output and -oG greppable output to -oX XML 
output. I have experimented with perl's Nmap::Parser and Nmap::Scanner, ruby's Nmap-Parser along with Tom's recently 
released Fathom, as well as XMLstarlet. I admit that I am fairly new to XML so there was a not insignificant learning 
curve for me to get up to speed with Nmap XML parsing in general. In fact, it is the desire to effectively parse and 
process Nmap XML files that prompted me to finally dig into XML. I recognize some major advantages with XML output such 
as integration with databases and feeding output in real time (or not) to other applications that can leverage the 
power of Nmap's capabilities without having to write sed, awk, grep, perl or other scripts and the maintenance 
headaches that accompany that approach. XML is a great and 
 established hook into other apps. One could say that XML output 'future-proofs' Nmap.

XML output should be the most robust output format. Ideally, XML output contains all of the information pertaining to a 
scan that one would need. However, there are a few key items that I have observed are not listed in XML output: 

* Host Status - i.e. If a host is down (or skipped), the record is not created in XML. Standard output shows this 
detail however. If the argument is that this will waste storage space with all the extra text, then enable the verbose 
-v (-vv) switch to print these records to -oX. This is a vital detail that really should be in -oX. I have numerous use 
cases for why I say this.

* Hostname PTR resolution status - If a rDNS record cannot be resolved, let -oX know about that. This omission causes 
Nmap::Parser to choke and spit out an error (eest on freenode recently submitted a patch to correct this after we 
identified the issue). Also, the hostnames  order when displayed in HTML (user and PTR) are often reversed 
inconsistently. May require a minor XSLT tweak.

* # of IPs - If a hostname has multiple IPs, standard output will indicate how many there were and Nmap proceeds 
scanning the first one found. This number is omitted from -oX. Ideally, all the IPs for a hostname will be listed in 
-oX, but at the very least, please include the # of IPs found. This is often an indicator of a load balancer which is a 
useful piece of information.

Also, as an XML newbie, it took me a long time to find a very helpful piece of information: If you want to look at your 
XML output in your browser, you have to transform it to HTML with xsltproc first like this:

$ xsltproc filename.xml > filename.html

Then just open filename.html in your browser and you will see beautifully formatted HTML results of your scan.

And of course, to create a Nmap xml file in the first place, you simply use the -oX switch:

$ sudo nmap -sV -p80 192.168.1.1 -oX filename.xml

On a final note, if you really want to convert the masses to using XML output, I have one final suggestion:

At the bottom of the html output screen, offer a "click to download in .csv format" button. This will get most users 
outputting to XML knowing that they can easily download a greppable file from their browser, and some will undoubtedly 
see the power of Nmap XML output and find ways to stick with it.

I am not an XSLT expert by any stretch, but I may take this challenge up myself and see if I can't update the 
stylesheet with this functionality and contribute to Nmap in some positive way(s).

Cheers,
Blake
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/