[NSE] http-enum and http-fingerprints enhancement suggestions

[Jesper Kückelhahn <dev.kyckel () gmail com>]

Hi List,

I really like the idea behind the http-enum script, and think it's nice for web app information gathering. I'd like to 
enhance the script (and fingerprint database) such that even more information can be extracted and included in the nmap 
output. In thread [1] there is a discussion about how this might be done, but I thought that I'd make a new thread 
(with a better title) where input and ideas could be collected. 

The http-enum script currently displays the results in the standard script output area regardless of whether the 
information found is version related. I'd like to modify the script and fingerprint file, such that it is possible to 
add fingerprinted web applications to the 'extra info' field of the ports table. I think that this extra information in 
the standard nmap output would make sense and would add value to the service detection. One of the strengths of 
http-enum is that it is able to find interesting url's such as '/doc/', '/upload/' and other generic urls, and this 
should be preserved. I think it would be interesting if the script (and fingerprint file) included a kind of type 
definition, which would allow it to be run as version gathering or comprehensive information gathering (generic urls, 
interesting files, etc). Furthermore an intrusive type could be added for none standard requests, methods, etc.

These modifications would require some general changes to the information stored in the fingerprint file. I've listed 
the properties I think would be needed/make sense to include:

  - type: [version, comprehensive, intrusive]
  - rarity: [?]
  - method: [HEAD, GET, OPTION]
  - favicon: md5 hash
  - page_hash: md5 hash
  - probe: url
  - match: regex
  - output: string
  - source: [Nikto, Blindelephant, ...]
  - cpe: cpe:/a:...

I've included favicon here as I think it would be a good way of centralising data, and possibly joining the scripts. 
Also adding a rarity field would allow for limiting the amount of data sent during the fingerprinting. This could be 
based on the popularity, port number, etc. I haven't figured out the best way of doing this, or if it's actually a good 
idea. The page_hash is inspired by BlindElephant[2] and [3]. I've added a source property in case fingerprints were 
included from other sources. 

I'd like to hear if anyone has comments, ideas, pro, cons etc for these changes, whether it would be worth the effort, 
if it would be used, etc. I'd personally like to see the version detection of web-apps be added as scan type, like -sC 
(-sW ?), but this might not be in line with the direction or guidelines of nmap. In either case, I think that a unified 
script for web-app detection and versioning would be a very useful enhancement.


- Jesper

------
[1]: http://seclists.org/nmap-dev/2013/q1/55
[2]: http://blindelephant.sourceforge.net/
[3]: https://secwiki.org/w/Nmap/Script_Ideas#Web_app_fingerprinting_.28static_files.29
[4]: https://secwiki.org/w/Nmap/Script_Ideas#vulndb

_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/