Re: New script: http-headers.nse

[Ron <ron () skullsecurity net>]

Thanks for your comments, they're great! I've taken care of everything, 
see inline comments.

On 08/27/2009 03:06 AM, Fyodor wrote:
> On Tue, Aug 25, 2009 at 06:37:48PM -0500, Ron wrote:
> > I went ahead and checked this in, with the change suggested by Patrick.
> > Let me know if there are any issues!
>
> Thanks Ron.  At first I thought the usefulness of this was a bit
> dubious considering how easy it is to do manually with ncat.  Then I
> though, "well, this makes it easy for SSL servers too", but actually
> that is equally easy with Ncat.  But after further contemplation, I do
> support the inclusion overall as http is an incredibly important
> protocol and many of these headers can actually be quite useful.  I
> have a few comments/questions though:
For me, I was doing some larger scans to figure out where http-enum and 
the like were breaking/returning false positives. Every time I saw one, 
I wanted to get the "Server:" header, and other information. Manually 
getting it with telnet/ncat/whatever was getting tedious, and I can see 
other people using this, despite its simplicity.

> o Why do a GET request instead of HEAD?  I'm not saying we should
>    switch to HEAD, just wondering about your reasoning.  After all,
>    HEAD exists for basically this exact purpose.  And it is slightly
>    less intrusive on the server and does not waste bandwidth giving us
>    content the script doesn't even look at.
Not all servers respond properly to HEAD. In other scripts, we try HEAD, 
and fail over to GET if HEAD's no good. So I originally opted to use a 
plain GET request to save effort.

I looked at my function http.can_use_head(), and it was actually really 
easy to modify it to implement this without a lot of overhead/extra 
code, so I did that. It now does a HEAD then, if that fails, a GET, the 
same way the rest of my scripts do.

On the GET fallback, I'd like to investigate using the "Range:" header 
to limit the amount of data returned.

> o The script loses the capitalization of the headers.  I guess that is
>    an http.get limitation.  If it was just as easy, I'd prefer to keep
>    the capitalization in http-headers.  But it may not be worth adding
>    some sort of option to http.get.
Fixed. I added an extra field to the returned 'data' for 'rawheaders', 
which contains the headers as they came, in order, no parsing.

> o The script also seems to lose the header order (presumably due to
>    http.get API).  If we're going to lose the original order, we should
>    probably just sort them rather than print them out semi-randomly.
>    Or they could be given in a canonical order specified in the script
>    which tries to group similar headers and also print more important
>    ones first.  Any which aren't specified in the script would be
>    alphebetized at the end.
Fixed, the same way as the previous one. It now preserves 
capitalization/order/anything else that http.lua parses automatically.

> o It would be great to have an NSE arg for specifying the path rather
>    than always grabbing /.  Sometimes I want to see the last modified
>    date or content type or some other values for a specific URL.
Added. Also added one that forces GET instead of HEAD.

> Cheers,
> Fyodor
Ron


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org