Re: Scripts for proxy detection

[Joao Correa <joao () livewire com br>]

Hello everyone,

I'm posting two new versions for the open proxy detection scripts.

The new features are:

HTTP:
* Changed pattern for connect (tested and supporting both polipo and
ncat, needs to test on squid!)
* Default test address is now nmap.org, and not www.google.com
* HTTP status codes 200, 301 and 302 are recognized as valid
responses, meaning that the proxy is correctly working
* It is also possible to use a different test address, specified with
script-args. The args should be "url", with the url that might be
tested and "hurl" with the url used to set the "Host:" field of the
HTTP requests. If no hurl is set, than url is used as hurl. If none is
set, nmap.org is used.
* Notice that the arguments will only work with nmap patched for
correct argument parsing

Socks:
* Default test address is now nmap.org, and not www.google.com
* Replaced function packet.hextobin for bin.pack
* HTTP status codes 200, 301 and 302 are recognized as valid
responses, meaning that the proxy is correctly working
* Replaced excessive prints for stdnse.print_debug
* It is also possible to use a different test address, specified with
script-args. The arg is only "url" here. If it is not set, nmap.org is
used.
* Also will only work if nmap is patched
* Tested on Dante and ssh

Questions:
Should we include new HTTP status codes?
Should we remove POST from the HTTP open proxy tests? (as mentioned by
David in his previous e-mail)
Socks script only tests a connection to the destination's port 80.
Should we include more ports? If yes, how should we parse the
responses? (we'll have different responses from each service)

Thanks everyone!
Joao

On Tue, May 26, 2009 at 11:40 PM, Joao Correa <joao () livewire com br> wrote:
> On Tue, May 26, 2009 at 8:03 PM, David Fifield <david () bamsoftware com> wrote:
> > On Sun, May 24, 2009 at 03:06:43AM -0300, Joao Correa wrote:
> > > I've been working on some scripts to detect open proxy servers. As a
> > > starting point I've made a few changes on the http-open-proxy script,
> > > including different methods beyond GET. I've also included POST, HEAD
> > > and CONNECT methods to it.
> >
> > Thanks, João, this looks good. Here are some notes on the HTTP script.
> >
> > I did a test of the HTTP open proxy script using Ncat and Polipo
> > (http://www.pps.jussieu.fr/~jch/software/polipo/).
> >
> > $ ncat -l --proxy-type http 3128
> > $ ./polipo proxyPort=8123
> > $ nmap --script proxy-open-http localhost -p 3128,8123 -d
> > PORT     STATE SERVICE    REASON
> > 3128/tcp open  squid-http syn-ack
> > |  proxy-open-http: Potentially OPEN proxy.
> > |_ Methods succesfully tested: GET POST HEAD CONNECT
> > 8123/tcp open  polipo     syn-ack
> > |  proxy-open-http: Potentially OPEN proxy.
> > |_ Methods succesfully tested: GET POST HEAD
> >
> > The script correctly found all methods open for the Ncat proxy, but
> > missed CONNECT for the Polipo proxy. It is because of the match pattern
> > in the script: "^http/1.0 200 ok". That's what Ncat returns, but Polipo
> > returns
> >
> > $ ncat -C localhost 8123
> > CONNECT www.google.com:80 HTTP/1.0
> >
> > HTTP/1.1 200 Tunnel established
> >
> > The match should work for any HTTP version number as long as the code is
> > 200, and should ignore the human-readable status message that follows.
> > Check http://tools.ietf.org/html/draft-luotonen-web-proxy-tunneling-01
> > and see if any other status codes should be reported. (There may not be
> > any, just check.
> Ok, I think I won't have any problem setting the correct regex for
> dealing with such pattern.
>
> > I haven't tested the SOCKS script yet. This is the only problem I found
> > in the HTTP script.
> No problem. As we already discussed on IRC, I'll soon change the calls
> to packet.hextobin to bin.pack. As soon as it is done I'll post it
> here.
>
> > > I was planning to include socks open proxy detection on the same
> > > script, but, as both proxy servers are different and use to run on
> > > different ports, I think it is better to set two different scripts,
> > > with different port rules. This way, http related code would never be
> > > sent to a socks proxy, what could happen if both tests were on the
> > > same script. A way of having only one script would be testing the open
> > > ports and test only those proxy servers related to such ports...
> > > anyway I think that this solution isn't the best, since it will make
> > > the code much more complex.
> >
> > I think this is the right idea. I support having two different scripts
> > for this, especially if the mechanism for testing is different.
> >
> > > The socks detection script is also displaying a lot of connection
> > > related information. I'm planning to soon make all these information
> > > visible only if requested by the user with a script argument.
> >
> > You can use the print_debug function, and then the information will only
> > be visible with -d, -d2, or whatever you choose.
> Very nice, I'll make such change also.
>
> > > I've also changed both script names because with the new names both
> > > scripts will be listed together in the script directory, what I think
> > > is nice, since they are both related. Anyway, the new names are not
> > > definitive.
> >
> > I would use the name http-open-proxy and socks-open-proxy to fit with
> > our general scheme of putting the protocol name first. It makes sense
> > for scripts that do the same thing to be together, but it also makes
> > sense for all the HTTP scripts, for example, to be together.
> The point is the http-open-proxy script isn't really a http script, I
> mean, all http scripts use to be thrown against port 80 or even 443,
> and to deal with HTTP specific issues... I'm not sure if it is the
> case for the http proxy, since it is a different port and a different
> kind of service.
>
> Socks-proxy and HTTP-proxy also don't run on the same port, but at
> least I think that both scripts are more related than http-open-proxy
> is with http-title, for example.
>
> Anyway, the name isn't that important, we can just change the name if
> it is the best.
>
> > Which SOCKS server did you use for testing? Is there a small one that's
> > easy to install that others can use to test this script?
> >
> > > If someone is able to test the code, I would really make good use of
> > > some feedback.
> >
> > Is it possible to use the http library
> > (http://nmap.org/nsedoc/modules/http.html) to do the GET test instead of
> > using comm.exchange? HEAD and POST are not possible yet but there is a
> > patch for them: http://seclists.org/nmap-dev/2009/q1/0889.html. CONNECT
> > may not be possible to handle in this way because it is not a normal
> > HTTP transaction.
> I'm not sure, since I didn't use the http library so long. Anyway,
> I'll try to do it. Should I apply the patch and try the HEAD and POST
> methods or should I wait until the patch is integrated with the trunk
> version? I don't feel that it is right to have a script that uses
> resources that are not yet completely integrated. Anyway, I don't know
> if the patch is being integrated or tested (Sorry, but I didn't follow
> any discussion about it)... but if it is about to become official, we
> could try to use it. Anyway, is there anything related to
> comm.exchange for us to not use it (besides http lib being a more
> specific resource)?
>
> > Maybe we shouldn't do a POST check because POST allows changing state on
> > the server. GET/HEAD/POST all belong to basically the same class anyway,
> > while CONNECT is something different. What do others think?
> I don't think that the POST tests are really making a big difference
> for now. Anyway, we have discussed that using google for the tests is
> privacy invasive, and that we should allow the tests on an argument
> defined url, what can start making a difference. Maybe we could make
> POST checkings disabled by default but possible to enable through an
> argument (or maybe disabled when using argument defined url and
> enabled when trying it on google/nmap.org).
>
> > What's the reason for using www.google.com.br in the CONNECT test when
> > www.google.com is used in the other tests?
> My mistake, no big reason. I'm just used to type google.com.br =)
>
> > David Fifield
> >
> > _______________________________________________
> > Sent through the nmap-dev mailing list
> > http://cgi.insecure.org/mailman/listinfo/nmap-dev
> > Archived at http://SecLists.Org
>
> Thanks a lot for the feedback David,
> I hope to be back soon with a new version
>
> João

Attachment: http-open-proxy.nse
Description:

Attachment: socks-open-proxy.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org