Re: [NSE] [patch] Big changes to http-enum.nse

[Bob Radvanovsky <rsradvan () unixworks net>]

Just out of curiosity, are you enumerating industrial automation systems and embedded devices with web front-ends to 
show their web servers?

So far, I have encountered quite a few...

-r


----- Original Message -----
From: Ron [mailto:ron () skullsecurity net]
To: nmap-dev () insecure org
Subject: [NSE] [patch] Big changes to http-enum.nse


> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey all,
>
> I spent today making improvements to http-enum.nse, and I'm happy to say
> that they're working perfectly. I'm attaching a .patch, but here's an
> overview of the changes:
>
> o I changed the http-fingerprints database to a new format. It no longer
> attempts to match the 'yokoso' project; instead, I made the database
> compatible with Nikto. Although we can't distribute Nikto checks without
> permission, this will let users use them anyway
>
> o The http-fingerprints database is now capable of searching for text within
> a page (either include all pages with text, or don't include any with text).
> Additionally, it uses Lua patterns and can perform captures, then display
> the captured data back to users. Although we don't have any checks yet, this
> can potentially give us powerful -sV-like functionality against HTTP servers
>
> o All pages found (with a 200 response) are stored in the registry. This
> will let later scripts use the pages as a seed for, for example, spidering
> (I'm hoping to start working on a primitive spider soon, too, that we can
> work off of). I haven't decided on the exact format for storing HTTP stuff
> in the registry yet, but it's a start. The two functions I put at the top of
> http-enum.nse can probably be moved to stdnse.lua in the future
>
> o On that topic, I moved get_hostname() from a local function in http.lua to
> a public function in stdnse.lua. I can see it being used for more than just
> internal http stuff. 
>
> o I updated the arguments to use more modern conventions
> ("http-enum.<argument>") and they're all read with stdnse.get_script_args()
> now
>
> o Added a more generic version of pGet() and pHead() to HTTP that can make a
> request with any verb. I updated pGet() and pHead() to use that function. 
>
> p Added response_contains() to http.lua. It searches the response's status
> line, headers, and body for the given string and returns success (any any
> captures) if it's found. 
>
> o Updated documentation all over the place. 
>
> For what it's worth, after combining my various fingerprint files, we have
> 999 web fingerprints. So close to an even number! 
>
> Against my server, and across the Internet, it takes only a few seconds
> (5-10) to do the whole thing. scanme.insecure.org doesn't seem to like
> pipelining, so it takes about a minute. Not terribly bad, in my opinion. 
>
> Anyway, let me know if you think it's good to commit. I'm hoping to start
> some preliminary work on http-spider.nse tomorrow, see how far I make it
> before I get stuck on something. :)
>
> Ron
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.16 (GNU/Linux)
>
> iEYEARECAAYFAky6jTIACgkQ2t2zxlt4g/QFZgCgshCqaa9QnzKoxoLDBBTE1Ftf
> qgYAoMBImVn8Cg5JNR6QVTBy5v1i6XLX
> =TKOA
> -----END PGP SIGNATURE-----
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/