Re: nmap+V

[Fyodor <fyodor () insecure org>]

On Tue, 22 Aug 2000, Max Vision wrote:

> Hi,
>
> I thought I should drop a quick note that I applied Saurik's
> "versioning" modifications (nmap+V) to the latest beta, available at:
> http://dev.whitehats.com/tools/netmap/nmap-2.54BETA3+V.tgz

I agree that this is incredibly cool.  Saurik has done impressive work
with his +V patch.  I would very much like to add protocol detection (and
possibly daemon versions as well) to Nmap.  This way Nmap can detect
exactly what service is running on a port rather than relying on the
static nmap-services lists.  Web servers, backdoor servers, and proxies
are some of the many applications that often listen on non-standard
ports.  Detecting protocols is the first step toward offering
protocol-specific probing.  For example, it would be nice if you could
configure a script to run whenever an http server is seen.  Also, tools
that use Nmap as their portscanning engine (Nessus, SAINT, SARA,
etc) would probably benefit greatly from knowing the REAL service running
rather than the port-based guess.

For more info on Saurik's work, see
http://lists.insecure.org/nmap-hackers/2000/Apr-Jun/0076.html .  To try
it, you should probably pick up Max's 2.54BETA3 update (URL up top).

I would love to add this, but I think we need some brainstorming first as
to the best format to use for the service detection logic.  Here are
some of my thoughts.  Ideas and proposals are VERY welcome!

-- The syntax needs to be powerful enough to handle the vast majority of
protocols.  Ideally, it could even handle binary protocols like
SMB.  Remember, it only needs to be smart enough to detect what protocol
is running (and perhaps version).  It doesn't need to do any in depth
analysis of the protocol.  The existing -sR can be used to detect RPC
protocols and determine the prog#/version running.  So you don't have to
worry about them.  But what other binary protocols might be hard to
detect?

-- The probe syntax should easily support TCP & UDP.

-- Adding version checking might be worth doing.  But it could potentially
add a lot of complexity to the grammer because you actually have to grab
(and possibly manipulate) information in the returned packets.  Still,
many ASCII protocols put out the version in a way that can easily be
captured by regular expressions.  On the other hand, look at the hoops
Saurik had to jump through to get IRC server versions (login, etc).

-- It needs to be fast, so I would rather have something that Nmap can
parse and handle itself in a parallel fashion rather than require perl/TCL
interpreters.  I would want convincing evidence of speed (and reasonable
download times and reasonable license) before using such a solution.  I
would probably be willing to include/require GLib (which includes a
lexer), and yacc/bison are fine (pcap uses them anyway).  I don't have a
problem with including a regexp library (like nmap+V includes).  I think
regular expressions are probably the way to go for matching.

-- The format needs to be simple to understand.  I want users to be able
to figure it out quickly so they can send me updates if one of their
servers is not detected.  That is one of my biggest problems with Nmap+V.  
It reads like line noise :).  Commands include '!', '&', and '*'.  The
asterict ('*') seems to mean "capture the given regular expression and
then spit back the values in parenthesized subexpressions to the server".  
The difference between the 'd', '&', and '!' commands is not at all
obvious without reading the code.

-- Ports should be used "hints" as to what typese of tests should be tried
first.  But services listening on non-standard ports (or standard ports
for other protocols) should still be detected.  For example, I think
Nmap+V will only detect AUTH servers if they are on port 113.  But tests
should include a lists of "likely ports" so that open ports will be routed
to the most likely port first.  Or maybe Nmap could look at the services
that can be detected by a particular test and correlate that with
nmap-services to determine which tests to do first.

-- Tests should be isolated as much as possible.  Ideally (from a
maintenance/accuracy standpoitn), each service would have a discreet test.  
But as a practical matter, we do not want to make 50 distinct connections
against each port to try each service test.  Many tests can be
combined.  For example, every service that announces itself upon connect
can be combined into one test which just connects and then matches the
service based on regular expressions.  Discreet tests allow easier
parallelisation and they scale better.  The current Nmap+V config file
( nmap-versions ) is basically one stream-of-control that contains
numerous "goto" statements to jump around to different sections.  Just
like code becomes spaghetti when you pack everything into one function and
jump all over the place, I think nmap-versions might encounter the same
scaling problems when you try to add support for thousands of server
applications running hundreds of protocols.

-- What are others doing?  Nessus has mentioned in various announcements
that they detect services rather than rely on static port mapping.  Has
anyone looked into their approach?  Sharing service detection
mechanisms/scripts with Nessus or other scanners would be a plus.  After
all, this is one of our key advantages with free software.  ISS & Cybercop
will never help each other out.  But Renaud and I have been cooperating
ever since he sent me an early beta months before Nessus was released.

I would be happy to hear other criteria for service detection or arguments
about the validity of the issues above.  Proposed service-detection
grammars would be cool.  A solution doesn't have to match EVERY one of our
criteria.  But I think we should make it as good as we can.  I would also
be more than willing to accept a patch implementing service
detection.  Maybe all we need is a few tweaks to Nmap+V.  Or maybe an all
new grammer is called for?  What do you guys think?  Please send your
thoughts to nmap-dev () insecure org .  I know there are a lot of smart
people on this list.

Cheers,
-F








---------------------------------------------------------------------
For help using this (nmap-dev) mailing list, send a blank email to 
nmap-dev-help () insecure org . List run by ezmlm-idx (www.ezmlm.org).