RE: nmap+V-2.0: (Partial) Protocol Auto-Detection !!

["Jay Freeman \(saurik\)" <saurik () saurik com>]

Alek:

Yeah, am handling that stuff (the strange responses from servers) through
the regular expressions by trying to focus on what is unlikely to change.
Not sure if you looked through the nmap-versions file, but I am running the
data I am getting through a large battery of tests.  Here is an example
(generated with -d3, at which point 2.0 will show the path of commands it
took to come to a decision) of scanning mail.linuxmall.com, which doesn't
even say what e-mail server it is running until you dig a little:

Key:
w Wait
< Read
> Send
* Bing (Bounce)
? Skip (If/Goto)
! Find
= End
p Port (If/Goto)
g Goto

w 1500000
< 128
Read: 220 Tux.LinuxMall.com ESMTP
? 1 ^$
? 2 ^220 .*FTP
? 3 ^220 .*SMTP
! 2 {} {} ^220 [^ ]+( ESMTP | )(Sendmail .+)(;| ready at)
! 1 {} {MS Exchange} ^220 [^ ]+ ESMTP Server \(Microsoft Exchange Internet
Mail Service (.+)\) ready
! 2 {} {MS SMTP} ^220 [^ ]+( Microsoft | )ESMTP MAIL Service, Version: (.+)
ready
! 1 {} {MS SMTP} ^220 [^ ]+ WindowsNT SMTP Server (.+) ESMTP ready
! 1 {} {} ^220 [^ ]+ ESMTP (Postfix)
! 1 {} {InterScan VirusWall NT} ^220 [^ ]+ InterScan VirusWall NT ESMTP (.+)
ready at
! 1 {} {} ^220 [^ ]+ ESMTP server \([^. ID.]*\)
! 1 {} {} ^220  (MailShield) SMTP
! 1 {} {} ^220 [^ ]+ SMTP (AnalogX Proxy .+) ready
> HELP\n
w 3000000
< 128
Read: 214 qmail home page: http://pobox.com/~djb/qmail.html
! 1 {} {} 214 (qmail)

Result: 25/tcp     open        smtp                SMTP         qmail

And here is a good one (removed the Read: lines from this one because they
get nuts) of scanning an IRC server (this one shows off some more advanced
features, including the random string generator (\R) and the * command,
which bounces part of a response back to the server (need that to handle the
PING from the server), and if you look in the config file, it makes heavy
use of decision branching):

w 1500000
< 128
? 1 ^$
p 101 80
p 101 8080
p 101 8888
p 101 5800
p 101 5801
p 101 5802
p 101 5803
p 101 5804
p 102 6665
p 102 6666
p 102 6667
> USER \R-----+ \R-----+ \R-----+ \R-----+\nNICK \R-----+\n
w 500000
< 512
? 201 PING
> PONG
* 1 PING ([^\r\n]+)
Bing: (:m4173254993)
> \n
w 500000
g 202
< 1024
! 1 {} {} :[^ ]+ 004 [^ ]+ [^ ]+ ([^ ]+)

Result: 6667/tcp   open        irc                 IRC
BeyondIRC-OMNI-4.0.2

I still haven't upgraded the some of the older entries in the configuration
file to use the more advanced techniques, but will be over the next couple
days (a few of the servers I built the original file off of didn't return
their versions, and was before I added the "default send HELP" to the SMTP
section of the file, so it just comes back saying it runs Sendmail even
though you CAN get the version).

As far as the machine-readable output, if you come up with a format that
would work well for you (just add to the end of existing entries and add
more /'s? escape the /'s (which are common in the version output) with
something? or is that going to require eliminating them and replacing them
with something ELSE which is escaped? am I going to have to deal with the
commas as well (seeing that people may be solving the problem of getting
extra fields separated by /'s by skipping to the first ,)... so far I
haven't seen any output with ,'s in it, but it is definitely a possibility
since I am just ripping part of the server's reply out and returning it... a
Server: header on a server with a , would definitely be a case of that) I
can start adding it.  I never use machine-readable output and I was wary of
breaking existing machine readers by just adding more fields to it before I
got responses from people who were already reading it (as was mentioned in
my first e-mail to the list).

Sincerely,
Jay Freeman (saurik)
saurik () saurik com



-----Original Message-----
From: Alek O. Komarnitsky (N-CSC) [mailto:alek () ast lmco com]
Sent: Wednesday, May 17, 2000 8:54 AM
To: nmap-hackers () insecure org; saurik () saurik com
Subject: Re: nmap+V-2.0: (Partial) Protocol Auto-Detection !!

> From: "Jay Freeman (saurik)" <saurik () saurik com>
> Subject: nmap+V-2.0: (Partial) Protocol Auto-Detection !!
> To: Nmap-Hackers <nmap-hackers () insecure org>
>
> nmap-type people:
>
> All right, this is the biggie :)!  I totally revamped the nmap-versions
> configuration file format to the point where it is almost like a
programming
> language.  Send this, read 128 bytes, if the data matches this regular
> expression then skip to section 3, if it matches this regular expression
the
> protocol is IRC, send logon information... etc.
>
> Not sure what to do next :-).  Definitely going to work on adding more
> protocols... Might take a look at what nnmap-web can do and see if there's
> anything it can do that my general system isn't good at, and then try to
> generalize it into the nmap-versions file... (not sure if I can generalize
the
> support for the time protocol, which is the one that was mentioned).


Jay,

What you are doing ROCKS ... assuming that Fyodor is willing to roll
this into the mainstream release, I'll pull the port version checking
functionality OUT of nmap-web ... it really belongs in nmap in the first
place.

nmap-web's port version checking is pretty simplistic - you have already
gone behond what it is capable of ... hopefully, the source code is pretty
clear in that area ... and yea, if you can generalize the time handling
stuff (both 13/daytime and 37/time - just different formats), that would
be awesome.

Oh yeah, nmap-web allows you to set an "expected" string to get back;
so this way, if you don't get it, it will highlight in RED ... this is
handy if for instance, I want to know which of the 1,000+ machines here
have NOT gotten upgraded to Sendmail8.9.3 (or which ones have more than
say, 15 seconds time deviation and therefore ntp isn't working correctly).
I think this functionality is probably more appropriate in nmap-web ...
since nmap is CLI based and one can use grep/etc. on that output.

I would be semi-careful about "expecting" a certain string ... for instance,
some people change the SMTP_GREETING to "NYB" (None of Your Business!  ;-)
and your code should be able to detect/display that string (?)

Finally, can you pls be sure to incorporate code so that when you call
nmap with the "machine readable" option (-oM), your stuff is generated?
nmap-web parses this since it is "easier" to parse and I figured less
prone to changes.

Very exciting stuff ... and yea, any competent IDS should notice this stuff;
but I think that is unavoidable.

Thanx,
alek