Re: ncat http proxy server and SSL

[David Fifield <david () bamsoftware com>]

(I forgot to include the list when I made this reply a week ago.)

On Sat, Feb 27, 2010 at 03:12:53PM +0100, Markus Klinik wrote:
> On Fri, Feb 12, 2010 at 05:37:50PM -0700, David Fifield wrote:
> > > As ncat_listen_stream already uses SSL, I'd like to reuse as much of
> > > that code as possible. In particular, there is the function "ncat_recv"
> > > which dispatches on whether SSL is compiled in and activated. It also
> > > does this SSL_pending thing. It does delaying, telnet negotiation and
> > > logging too.
> > >
> > > So, here is the plan:
> > >
> > > - replace socket_buffer.sd with an fdinfo
> > > - make socket_buffer_readline use ncat_recv instead of recv
> > > - make handle_connect and do_transaction use ncat_recv and ncat_send
> > >   where appropriate
> > > - put SSL init / deinit where appropriate
> > > - finally, as telnet negotiation doesn't make much sense in proxy mode,
> > >   forbid that combination in ncat_listen_mode
> > >
> > > What do you think, does that sound reasonable?
> >
> > That sounds reasonable. However, rather than forbidding Telnet
> > negotiation at a high level, I would rather see another function
> > factored out that does the SSL stuff without the other things that
> > ncat_recv does.  (It still probably makes sense to prohibit Telnet
> > with the HTTP proxy at a high level, but it should be enforced at a
> > low level to ensure the correctness of the HTTP traffic.)
>   
> I don't understand that. One of the purposes of ncat_recv is to dispatch
> between SSL and non-SSL. What is there that can be factored out?

The ncat_recv function does 1) a socket read, 2) line delay, 3) telnet
manipulation, and 4) logging. It's not really a low-level function. All
those are combined, only because before ncat_recv was introduced, all
four of those things always appeared together. Now we have a need for a
function that only does the socket read (and possibly logging, I'm not
sure). I would factor those out into an ncat_recv_primitive (compare the
existing ncat_send_primitive). The factored-out code wouldn't need the
special pending handling either.

> Also, as ncat_recv is a low-level function, it is unaware of the
> protocol. We could add a flag for ncat_recv to skip telnet stuff, and
> set that flag whenever we call ncat_recv from within HTTP code. But as
> the program options are some sort of global status anyway, just
> forbidding the combination at program start is enough and keeps the
> low level interfaces clean.

I must disagree; I don't consider it a clean design for low-level
functions to depend so critically on global state. However I think this
is really a minor issue, because I am impressed with the patch overall.

> Well, here is a patch set which enables SSL support in http-proxy mode.

> The CONNECT method works, tested with ssh and proxytunnel.  I couldn't
> test whether the other methods (GET,PUT,HEAD) work because I didn't find
> an http client that can connect to an SSL encrypted proxy.  Neither
> firefox nor wget seem to support SSL encrypted proxies. Is there some
> reason? Am I not getting why this might be undesirable or impossible?

I haven't found any web clients that support SSL connections to the
proxy. It's a shame, because certificates would provide reasonable
authentication (even Basic cleartext passwords over SSL would be all
right). So far, the only use case for an SSL-enabled proxy is for Ncat
clients to connect through it.

You can test GET and others with Ncat itself.

$ ncat -l --proxy-type http --ssl
$ ncat --ssl localhost scanme.nmap.org 80
GET http://scanme.nmap.org/ HTTP/1.0

There are already existing tests for SSL support in the proxy in
test/ncat-test.pl. They are currently marked as XFAIL to indicate that
the feature is unimplemented. It looks like this now:

PASS HTTP proxy server auth challenge (tcp)
XFAIL HTTP proxy server auth challenge (tcp ssl xfail)
PASS HTTP proxy server correct auth (tcp)
XFAIL HTTP proxy server correct auth (tcp ssl xfail)
PASS HTTP proxy Basic wrong user (tcp)
XFAIL HTTP proxy Basic wrong user (tcp ssl xfail)
PASS HTTP proxy Basic wrong pass (tcp)
XFAIL HTTP proxy Basic wrong pass (tcp ssl xfail)
PASS HTTP proxy Basic correct auth, different case (tcp)
XFAIL HTTP proxy Basic correct auth, different case (tcp ssl xfail)
PASS HTTP proxy Digest wrong user
PASS HTTP proxy Digest wrong pass
PASS HTTP proxy Digest correct auth
PASS HTTP proxy Digest correct auth, no qop
PASS HTTP proxy Digest missing fields
XFAIL HTTP proxy Digest prevents replay
PASS HTTP proxy server LWS (tcp)
XFAIL HTTP proxy server LWS (tcp ssl xfail)
PASS HTTP proxy server LWS (tcp)
XFAIL HTTP proxy server LWS (tcp ssl xfail)
PASS HTTP proxy server no auth (tcp)
XFAIL HTTP proxy server no auth (tcp ssl xfail)
PASS HTTP proxy server broken auth (tcp)
XFAIL HTTP proxy server broken auth (tcp ssl xfail)
PASS HTTP proxy server extra auth (tcp)
XFAIL HTTP proxy server extra auth (tcp ssl xfail)

Ideally, your patch should change all those XFAIL into PASS. I would
appreciate it if you would modify ncat-test.pl and see if the tests
pass. All you have to do is remove the "xfail" in lines like this:

server_client_test_multi ["tcp", "tcp ssl xfail"], "HTTP proxy server auth challenge",

> As I changed the interface of socket_buffer, I needed to adapt an
> unrelated part of the code, too (ncat_connect). I'm not particularly
> happy with that because it never makes use of the SSL feature of fdinfo
> (it just grabs the socket descriptor from fdinfo when proxy negotiation
> is finished), but it works, tested with an http proxy. Couldn't get a
> connection to any socks4 proxy, neither with or without my patch.

The ncat_connect.c changes look fine to me. That's disconcerting about
the SOCKS mode; we don't have tests for that. Anyway, please try the
tests and see if they work. Then please send another patch with the test
changes (a combined patch; I'm not used to dealing with the separate
patch files). I know I made it sound like the ncat_send stuff is a big
deal, but really I think it's very minor and I don't mind accepting the
patch as you've submitted it.

I'd be interested in your thoughts on the implementation of SSL in the
proxy client. Ncat might be the only client that can actually make use
of an SSL-enabled proxy.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/