Re: general scanning engine - request for comments :)

[Fyodor <fyodor () insecure org>]

On Wed, Jul 12, 2006 at 09:49:37PM +0200, majek04 wrote:
> 1) AVAILABLE PROTOCOLS AND THEIR CHARACTERISTICS:
>
> Columns:
> 'binary' - support for binary connections; tunneling
> 'dns'    - resolving dns names on remote site
> 'k-a'    - keep-alive; one connection, many probes
>
> ------------------|-binary-|-udp-|-dns-|-ipv6-|-k-a-
> HTTPPROXY CONNECT-|---Y----|-----|--Y--|--?---|-----
> HTTPPROXY GET-----|--------|-----|--Y--|--?---|--Y--
> SOCKS 4-----------|---Y----|--Y--|-----|------|-----
> SOCKS 4a----------|---Y----|--Y--|--Y--|------|-----
> SOCKS 5-----------|---Y----|--Y--|--Y--|--Y---|-----
> FTP BOUNCE--------|--------|-----|-----|------|--Y--
> classic connect()-|---Y----|--Y--|--Y--|--Y---|-----

Looks good.  I assume classic connect() is the Nmap connect scan
(-sT)?  If so, it doesn't really do "resolving dns names on the remote
site", nor does Nmap support UDP any longer using connect (though it
theoretically could).

Maybe it would be useful to implement connect scan through your proxy
scanning engine anyway as a simple case for testing/debugging/etc the
system.  It probably wouldn't become the default implementation of
-sT, but it would be interesting to compare the performance and timing
between the two implementations.

>       desc connect(desc, host, port, protocol)
>       query(desc, host, port, protocol)
>       parse(desc)

These makes sense.  Though as you noted in a later email, it may not
be quite so simple do to the asynchronous nature of the system.

> 3) EXAMPLE OF BUILDING A CHAIN
> Let's imagine that we have array with protocols:
> chain[]={CLASSICCONNECT, SOCKS, HTTPPROXY-GET}

If we can chain different systems and it doesn't complicate/bloat
everything too much, I'm all for it.  But as long as we can support
chaining of a single type which supports it (e.g. multiple SOCKS
proxies), that is probably sufficient.

> 4) PROBLEMS AND FUTUREWORK
> Dns resolving:
>       Some protocols can resolve dns for us. For example we can
>       send dns entry to http proxy.
>       But by default all targets are resolved in main nmap's functions.
>       There are also some cases when we need to resolve target locally.
>       For example if user specifies google.com/28.
>       I don't have clear idea how to handle these problems.

If you find an elegant way to handle this, go for it.  Otherwise, I
think keeping forward DNS resolution as is for now is OK.  But I can
definitely see us possibly wanting to add remote host DNS in the
future, so do try to keep that in mind.  We may want to give a privacy
warning message if the user DOESN'T specify -n.

> Ping probes:
>       Normally nmap is doing ping probes before scanning.
>       How such ping probes should look like when someone is using
>       proxy/socks chaining?

Maybe they can just use -P0.  I tend to think another warning message is warranted here if they use proxy scan but 
don't specify -P0.

> Timing issues:
>       The question is: how much time should we wait before saying
>       that port is filtered. This can be complicated when we'll have
>       many hops in chain.
>       Again, I don't have clear concept how to handle this.

Nmap has some good algorithms (often cribbed from TCP itself) for
handling this sort of situation.  The main idea is to start slow and
speed up as you determine that you can.  Once you get a few responses,
you can use the average time they took, along with the variance to
compute good timeout values.

> Service detection:
>       Imagine Version Detection through TOR or other anonymous proxy.
>       I think this could be really powerful tool.

Yeah.  I think we should try to isolate the proxy chaining code as
much as possible so that it can be more easily reused in ncat,
possibly version detection, etc.  Doug suggested putting the code in
Nsock.  That may make sense (especially if there are efficiency
advantages to integrating them), though isolating them in their own
files in Nmap with a reasonably generic API and as few (if any)
dependencies on Nmap stuff (like the NmapOps structure) might be a
good compromise.

I think this will be quite a valuable system!

Cheers,
-F


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev