RE: Nmap Service Detection Proposal

["Jay Freeman \(saurik\)" <saurik () saurik com>]

Fyoder:

Actually, I came across a problem with Xerces... I totally forgot it was C++
and not C :(, so people will be unlikely to want to use it (although it
would be easy to, as the new nmap+V patch I am going to put on my FTP server
later today compiles in g++, I had forgotten that the nmap-c++ patch was a
post-2.1 version of nmap+V... need to finish a few things that required an
internet connection).  The XSLT would definitely be useful as I am going to
explain in a reply to "XML Questions" (was going to send it in another
e-mail as a response to one of your earlier XML related, typed a bunch of
stuff last night, and then my cable people went down for maintenance...
finally gave up and dialed on with a good old fashioned phone based ISP).

I'm going to see what libxml supports in the next few days.  The hope is it
would support SAX, as that should be much easier for outputting the XML than
using DOM (although if the document is going to sit in memory and slowly be
edited by a bunch of modules as was the case in nmap++, DOM would likely be
a better choice).

I don't know if I would assume that just because the first couple characters
out of a port happen to be 220 it is FTP... I'd definitely want to make sure
some more of the line looks right... (maybe mentions FTP in it).  I couldn't
do that on a lot of the servers until I got a really large listing as the
server banner came after the message of the day.

Jumping:  What if I connect to a port, send GET /, and then WHAM, it realize
it is some server I know about.  Instead of trying a bunch of different
scans, I can now immediately skip to detecting that server.  BUT, I might
have just totally lost the ability to get the version by sending a GET /, as
the server might go into an error or quit state after getting undefined
information.  I would then want to disconnect and reconnect to the server to

Sincerely,
Jay Freeman (saurik)
saurik () saurik com

-----Original Message-----
From: Fyodor [mailto:fyodor () insecure org]
Sent: Sunday, August 27, 2000 5:09 AM
To: Jay Freeman (saurik)
Cc: Nmap-Dev
Subject: RE: Nmap Service Detection Proposal

On Sun, 27 Aug 2000, Jay Freeman (saurik) wrote:

> Unless
> there is some hope that a large number of people out there already have
> libxml,

Actually, there may be some hope!  Look what I just found on my system:

amy~>ls -ld /usr/lib/libxml.a /usr/lib/libxmlparse.a
-rw-r--r--    1 root     root  402640 Feb  4  2000 /usr/lib/libxml.a
-rw-r--r--    1 root     root  25034  Aug 12  1999 /usr/lib/libxmlparse.a

I don't remember why I installed them (or they could have come with
Redhat).  I could ask rpm, but my point is that libxml may be more common
than we think.

> Xerces is a complete implementation of DOM and SAX and all sorts of
> other XML goodies, plugs into an XSLT container well (Xalan,

Do we need these?  Whatever they are :) ?

> I do like the idea of using XML as the base file library, as it solves all
> of the escaping issues (most, if not all, of the escaping is done by the
XML
> parser).

Yeah that is nice.  And it could make future Nmap file formats easier to
parse.  But adding a required library dependancy to Nmap is not something
I take lightly.  Tough call.

> I'm going to sit down sometime tomorrow (assuming I have some time, I
think
> I do... have to work on a document with my partner, but that shouldn't
take
> _that_ long) think of different ways to handle the jumping issues (if we
> think its HTTP, and fail, but now know it is some other protocol, but have
> to start over again, we know what kind of connection to jump to), and ways
> of using the ports for sorting help without being locked in by them at the
> same time.

I'm not sure I understand the need to jump.  With my latest proposal, the
idea is:

1) find port XX open
2) execute the probe(s) which registered that port (possibly in parallel)
3) If the registered probes fail, execute the other tests (possibly in
   parallel) until one succeeds.

Could you give examples of cases where you think jumping would be a big
help?

> A few issues that need to be dealt with, however, are stuff like timeouts.
> Some services are just slower than others.  I noticed this while building
[ ... ]
> 4 seconds I am going to get a reply, an example (I think, was a while ago)
> was sending a HELP command to a mail server to get more accurate/further
> version information

Well, this particular case wouldn't be neccessary if you were only looking
for service type (and not version info).  But you are probably right that
some services may take particularly long even for the initial response we
need for service-detection.  If we need it, adding an optional timeout
attribute to the probe line should not be a problem.

> Even if you were to try to give the daemon as much time as possible and go
> on to other things, the data length issue is still a problem.  In the case
> of FTP I use a 4096 byte receive buffer (< 4096) to get data because
> sometimes I couldn't get the version string until long after the initial
> connection as people were putting long connection banners on their FTP

This is another case where trying to get the version # is causing you
pain.  If you were only doing service detection, you would know by the
'^220 ' on the first line.

But your point is still very valid.  If the text we get back does not
match any of the expressions, it would be nice to know when we have
collected enough data to give up (rather than waiting for the timer to
expire).  Perhaps we should add attributes to the probe line specifying
the amount of data (in lines and/or bytes) that we have to get before we
can give up early.

Cheers,
-F


---------------------------------------------------------------------
For help using this (nmap-dev) mailing list, send a blank email to 
nmap-dev-help () insecure org . List run by ezmlm-idx (www.ezmlm.org).