[NSE] [patch] Big changes to http-enum.nse

[Ron <ron () skullsecurity net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey all,

I spent today making improvements to http-enum.nse, and I'm happy to say that they're working perfectly. I'm attaching 
a .patch, but here's an overview of the changes:

o I changed the http-fingerprints database to a new format. It no longer attempts to match the 'yokoso' project; 
instead, I made the database compatible with Nikto. Although we can't distribute Nikto checks without permission, this 
will let users use them anyway

o The http-fingerprints database is now capable of searching for text within a page (either include all pages with 
text, or don't include any with text). Additionally, it uses Lua patterns and can perform captures, then display the 
captured data back to users. Although we don't have any checks yet, this can potentially give us powerful -sV-like 
functionality against HTTP servers

o All pages found (with a 200 response) are stored in the registry. This will let later scripts use the pages as a seed 
for, for example, spidering (I'm hoping to start working on a primitive spider soon, too, that we can work off of). I 
haven't decided on the exact format for storing HTTP stuff in the registry yet, but it's a start. The two functions I 
put at the top of http-enum.nse can probably be moved to stdnse.lua in the future

o On that topic, I moved get_hostname() from a local function in http.lua to a public function in stdnse.lua. I can see 
it being used for more than just internal http stuff. 

o I updated the arguments to use more modern conventions ("http-enum.<argument>") and they're all read with 
stdnse.get_script_args() now

o Added a more generic version of pGet() and pHead() to HTTP that can make a request with any verb. I updated pGet() 
and pHead() to use that function. 

p Added response_contains() to http.lua. It searches the response's status line, headers, and body for the given string 
and returns success (any any captures) if it's found. 

o Updated documentation all over the place. 

For what it's worth, after combining my various fingerprint files, we have 999 web fingerprints. So close to an even 
number! 

Against my server, and across the Internet, it takes only a few seconds (5-10) to do the whole thing. 
scanme.insecure.org doesn't seem to like pipelining, so it takes about a minute. Not terribly bad, in my opinion. 

Anyway, let me know if you think it's good to commit. I'm hoping to start some preliminary work on http-spider.nse 
tomorrow, see how far I make it before I get stuck on something. :)

Ron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAky6jTIACgkQ2t2zxlt4g/QFZgCgshCqaa9QnzKoxoLDBBTE1Ftf
qgYAoMBImVn8Cg5JNR6QVTBy5v1i6XLX
=TKOA
-----END PGP SIGNATURE-----

Attachment: http-enum-changes.patch
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/