Updates to http-enum.nse

[Ron <ron () skullsecurity net>]

Hi all,

Me and one of my minions at work (Andrew -- same guy who I did the iis 
unicode script with) have put a lot of work into improving http-enum.nse 
(in case that wasn't obvious from all the http.lua errors I've been 
posting). Rob's script was a great start, but we made a ton of improvements:
- Cleaned up the code, put a bunch of it into functions
- Support for many more HTTP status codes
- Improved detection for 404 pages (especially those that return 200) -- 
we still have some more work to do on this, but it's getting there
- More intelligent usage of HEAD vs. GET requests
- Ability to parse external fingerprint file (attached)

That last point is the interesting one, to me -- we use the same file 
format as the Yokoso project (by Kevin Johnson and others, from Intel 
Guardians). This lets us leverage their fingerprints as well (and 
they've given me permission to include a copy of their fingerprints 
file, too, and they'd like to include the .nse script with Yokoso once 
it's ready, which I told them is fine). For those who were there (and I 
know several of you were, because I was sitting with you :) ), there was 
a presentation about Yokoso at Defcon.

Anyway, the intent of the Yokoso fingerprints is to identify Web 
applications running in standard locations by finding standard files 
(such as images, javascript, php files, etc) associated with it. It 
works very nicely with the http-enum.nse code. See the 'fingerprints' 
file attached for more info.

One thought I had -- http-enum.nse and Yokoso sort of have different 
points. http-enum.nse is designed for finding common locations, like 
/icons, /scripts, /test, etc, and Yokoso is designed for fingerprinting 
common web apps. So, for that reason, it might make sense to put it in a 
different script that the user can run separately. Or maybe not. I'm 
happy with going either way.

I plan to move the hardcoded tests from http-enum.nse into their own 
file, too, once I'm happy with how it's working. For now, I wanted to 
make it easier to test this without the fingerprint file installed (if 
you want to try it, put the attached 'fingerprints' file in 
nselib/data), but the script will run fine without it, it'll just find 
less cool stuff.

So, please let me know what you think, and any issues you have! I'd like 
to commit this to SVN if everybody is ok with it.

Known issues:
- 404 pages that return 200 and dynamic content (like a random haiku)
- chunked encoding is causing an occasional error
- pipelined requests to certain servers are acting funny

Fixes for those are in the works.

Thanks!
Ron

Attachment: fingerprints
Description:

Attachment: http-enum.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org