Re: http redirect problems

[Patrik Karlsson <patrik () cqure net>]

On Thu, Feb 28, 2013 at 7:37 PM, Robin Wood <robin () digininja org> wrote:

> On 1 March 2013 00:11, Patrik Karlsson <patrik () cqure net> wrote:
>
> > On Thu, Feb 28, 2013 at 12:38 PM, Robin Wood <robin () digininja org> wrote:
> >
> > > I'm trying to get the redirect_ok closure working with http.get but
> > > having
> > > problems so looking for some help.
> > >
> > > This is the code:
> > >
> > >     result = http.get(host, port, path, { redirect_ok =
> > > function(host,port)
> > >             stdnse.print_debug(1, "********** REDIRECT ************")
> > >             local c = 5
> > >             return function(url)
> > >                 stdnse.print_debug(1, "********** " .. url.port .. "
> > > ************")
> > >                 stdnse.print_debug(1, "********** " .. url.host .. "
> > > ************")
> > >                 if ( c==0 ) then return false end
> > >                     c = c - 1
> > >                     return true
> > >                 end
> > >         end
> > >         }
> > >     )
> > >
> > > If I start the process with just twitter.com then it redirects to
> > > https://twitter.com. The redirect code kicks in but then I get bounced
> > > from
> > > http to https and back again till the counter hits 0 and the script
> > > returns
> > > false. This is the output
> > >
> > > NSE: ********** REDIRECT ************
> > > NSE: ********** 443 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** 80 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** 443 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** 80 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** 443 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** 80 ************
> > > NSE: ********** twitter.com ************
> > > NSE: ********** AFTER GET ************
> > >
> > > I checked by hand and if I curl http://twitter.com then it shows the 301
> > > redirect but if I then get https://twitter.com it returns a page, not
> > > another redirect.
> > >
> > > Have I done something wrong here?
> > >
> > > Robin
> > > _______________________________________________
> > > Sent through the dev mailing list
> > > http://nmap.org/mailman/listinfo/dev
> > > Archived at http://seclists.org/nmap-dev/
> >
> > Robin,
> >
> > I tracked the issue down to the host header of the request.
> > What happens is that nmap adds the port number to the host header when
> > redirected to SSL, while wget and curl both don't.
> > This seems to confuse the host your testing against (twitter.com) which
> > then goes into a loop.
> > A cursory glance at the HTTP/1.1 RFC suggest adding the port in the host
> > header is correct.
> > So I'm not sure what to do with this.
> Glad it isn't something that I've done wrong. I'm going to be scanning a
> load of sites from the Alexa list so may come across more broken sites.
>
> For now I'll just treat it as unscanable as I'm not to worried about
> missing the odd site, if you work out a way to fix it then let me know.
>
> Robin
>
>
> > Cheers,
> > Patrik
> >
> > --
> > Patrik Karlsson
> > http://www.cqure.net
> > http://twitter.com/nevdull77
Robin,

I believe this patch will fix it, if you temporarily want to patch the http
library before conducting your scan.
As for a permanent fix, I would be interested in what others think would be
a suitable approach.

--- nselib/http.lua (revision 30634)
+++ nselib/http.lua (working copy)
@@ -1106,7 +1106,7 @@
   local mod_options = {
     header = {
       Connection = "close",
-      Host = get_host_field(host, port),
+      Host = stdnse.get_hostname(host),
       ["User-Agent"]  = USER_AGENT
     }
   }

Thanks,
Patrik
-- 
Patrik Karlsson
http://www.cqure.net
http://twitter.com/nevdull77
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/