Unused captures in nmap-service-probes

[David Fifield <david () bamsoftware com>]

I've been working on the sv-tidy.py script to canonicalize and find
errors in the nmap-service-probes database. Committers have access to
this script in /nmap-private-dev/misc-scripts and I'm also attaching a
copy as of r27890.

I added a feature to warn when a capture is present in a pattern but not
used in a template. However, some of these are errors and some are not.
I want to ask if there's a volunteer willing to go through the list and
fix the ones that are errors. Doing this is definitely enough to get you
in the CHANGELOG.

For example, one line of output is
        281: unused capture $1 (of 1)
Line 281 of nmap-service-probes is
        match daytime m|^[A-Z][a-z]+day, [A-Z][a-z]+ \d{1,2}, \d{4} \d\d:\d\d:\d\d-\w\w\w(-DST)?\r\n| p/Cisco router 
daytime/ o/IOS/ cpe:/o:cisco:ios/a
The capture (-DST) is not used, but that's because it's only being used
for grouping, not as a capture, so this is not an error. You fix these
cases by putting ?: at the beginning of the group, for example (?:-DST).

No, the other hand, the output
        513: unused capture $3 (of 3)
is a real error. Line 513 is
        match ftp m|^220 ([-.\w]+) FTP server \(Version (\S+) VFTPD, based on Version (\S+)\) ready\.\r\n$| p/Virtual 
FTPD/ v/$2/ i/based on $2/ o/Unix/ h/$1/
Here we see that $2 is used twice but $3 is not used. The i// template
should be instead i/based on $3/.

Just run the script like this and fix errors until all the obvious ones
are gone:
        ./sv-tidy.py nmap-service-probes -n --unused-captures

Some cases might not be trivial to handle. For example, line 402:
        match ftp m|^220 ([-\w]+)? FTP version 1\.0 ready at | p/Netgear broadband router or ZyXel VoIP adapter ftpd/ 
v/1.0/
The first capture looke like it should be a host name (h/$1/), but I'm
not sure what happens when it's not present because of the ? that
follows it. Probably the right thing to do is to break this into two
match lines, one with the host name and one without. But you can leave
any that you're not sure of.

David Fifield

Attachment: sv-tidy.py
Description:

Attachment: unused-captures-r27889.txt
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/