Re: Proposed SSL version detection probe changes

[Kristof Boeynaems <kristof.boeynaems () gmail com>]

On Sun, Feb 8, 2009 at 7:06 PM,  <doug () hcsw org> wrote:
> Nice. You may have already read this but version detection handles
> SSL specially:
>
> http://nmap.org/book/vscan-post-processors.html#vscan-ssl-postprocess
>
> The idea is that if with probing we detect that a port is SSL, then
> we open up a real SSL connection with OpenSSL and run version detection
> through that.
>
> Any improvements to the SSL probes would be great. I'm just now
> processing a bunch of fingerprints and there are always a few
> SSL ones that don't get properly recognized.

Hi Doug, All,

I had a deeper look at the SSL version probing, and would like to 
suggest the following:

Instead of trying to fingerprint the SSL response, I think it is more 
robust to simply *recognize* SSL, and then reconnect with an SSL probe
to get more information on the application behind SSL.

This can be done as follows:

1. Add/rewrite the SSL version probes to reliably detect *all*
SSL-enabled services in a generic way, as well as the specific SSL
version supported (this information is needed to connect with the
correct SSL version later on).

2. Rewrite the SSL connection engine to take the exact SSL version
detected by the probes (e.g. "sslv2", "sslv3" or "tlsv1"), and create
the correct SSL connection (that is, SSLv2-compatible (SSLv23),
TLSv1-only or SSLv3-only).
An additional requirement for this step is that the extended SSL
connection support can also be integrated with Ncat, of course (as
this is how it all started, see
http://seclists.org/nmap-dev/2009/q1/0319.html ;))

Any thoughts on this approach?


The first step is the easiest, which I already implemented myself,
based on the probes I listed earlier. See attached 
"nmap-services.probes.patch", a patch against the
"nmap-service-probes" file that comes with Nmap 4.76.
In summary, I commented out the original SSLv3SessionReq probe, and
instead defined following two SSL probes:

- SSLv23SessionReq, which will sent out a SSLv2-compatible
ClientHello. This will match all SSL servers, apart from a SSLv3-only
and TLSv1-only server, and reliably detect the SSL version used
("sslv2", "sslv3" or "tlsv1").

- TLSv1SessionReq probe, which will sent out a TLSv1 ClientHello. This
will match a SSLv3-only or TLSv1-only server, again reliably detecting
the SSL version used ("sslv3" or "tlsv1"). This Probe does not come
with any matches, but will instead "fall back" on the matches in
SSLv23SessionReq.

I based the matches on OpenSSL testing, but I tried to make the match
lines as generic as possible; I believe that they will correctly match
all possible (RFC compatible) SSL implementations.
For now this Probe detection process returns "sslv2", "sslv3" or
"tlsv1" (instead of the generic "ssl"). It is possible to
differentiate even further between e.g. "sslv3-only" and "sslv3", if
people would be interested in that.

Note that I also added the port 4433 to these new probes, as this is
the default port of "openssl s_server".


The second step is a bit more work, and before undertaking such an
effort, I'd like to get your feedback on this approach first.

Additionally, if anyone with some sound Nmap coding (possibly SSL
related) or just plain enthusiasm is willing to team up with me on
this one, please let me know!

Note that this is quite a major change, which will most likely break
with all existing SSL fingerprints. However, I believe it will
ultimately make the detection of SSL-enabled services a lot more reliable.
If you have a good idea to stay backward-compatible with the existing
fingerprints, let me know.

One good use of the existing SSL fingerprints (or SSL fingerprints in
general) I still see, is in case Nmap is compiled without OpenSSL
support. In these cases it will not be able to SSL-connect to the
service, and the information collected by the probe is all we can act
on. Nevertheless, even in this case, I believe the new probes will
trigger more useful (extensive) SSL fingerprints.
However, I do not immediately see how this non-OpenSSL-compatibility
can easily be implemented. Maybe we can provide a different
nmap-service-probes file, depending on whether OpenSSL support is
enabled or not?

I welcome all comments.

Thanks,

Kristof

--- nmap-service-probes.orig    2009-02-09 18:21:09.000000000 +0100
+++ nmap-service-probes 2009-02-09 18:17:10.000000000 +0100
@@ -6048,67 +6048,105 @@
 match webster m/^DICTIONARY server protocol:\r\n\r\nContact name is/ p/Webster dictionary server/
 
 ##############################NEXT PROBE##############################
-Probe TCP SSLSessionReq 
q|\x16\x03\0\0S\x01\0\0O\x03\0?G\xd7\xf7\xba,\xee\xea\xb2`~\xf3\0\xfd\x82{\xb9\xd5\x96\xc8w\x9b\xe6\xc4\xdb<=\xdbo\xef\x10n\0\0(\0\x16\0\x13\0\x0a\0f\0\x05\0\x04\0e\0d\0c\0b\0a\0`\0\x15\0\x12\0\x09\0\x14\0\x11\0\x08\0\x06\0\x03\x01\0|
+#SSLv2-compatible ClientHello, 39 ciphers offered
+#Will sollicit a ServerHello from most SSL implementations, apart from the ones that are TLSv1-only or SSLv3-only
+Probe TCP SSLv23SessionReq q|\x80\x9e\x01\x03\x01\x00u\x00\x00\x00 
\x00\x00f\x00\x00e\x00\x00d\x00\x00c\x00\x00b\x00\x00:\x00\x009\x00\x008\x00\x005\x00\x004\x00\x003\x00\x002\x00\x00/\x00\x00\x1b\x00\x00\x1a\x00\x00\x19\x00\x00\x18\x00\x00\x17\x00\x00\x16\x00\x00\x15\x00\x00\x14\x00\x00\x13\x00\x00\x12\x00\x00\x11\x00\x00\n\x00\x00\t\x00\x00\x08\x00\x00\x06\x00\x00\x05\x00\x00\x04\x00\x00\x03\x07\x00\xc0\x06\x00@\x04\x00\x80\x03\x00\x80\x02\x00\x80\x01\x00\x80\x00\x00\x02\x00\x00\x01\xe4i<+\xf6\xd6\x9b\xbb\xd3\x81\x9f\xbf\x15\xc1@\xa5o\x14,M
 \xc4\xc7\xe0\xb6\xb0\xb2\x1f\xf9)\xe8\x98|
 
-rarity 3
-ports 443,444,548,636,993,1241,1311,2000,4444,5550,7210,7272,8009,8194,9001
+rarity 3 
+ports 443,444,548,636,993,1241,1311,2000,4433,4444,5550,7210,7272,8009,8194,9001
 fallback GetRequest
 
+# SSLv2 ServerHello
+match sslv2 m|^..\x04\0.\0\x02|
+
+# TLSv1 ServerHello:
+match tlsv1 m|^\x16\x03\x01..\x02...\x03\x01|
+
+# SSLv3 ServerHello:
+match sslv3 m|^\x16\x03\0..\x02...\x03\0|
+
+
+##############################NEXT PROBE##############################
+#TLSv1 ClientHello. Will sollicit a response from both SSLv3-only and TLSv1-only servers; that is, the servers that 
are not covered by the SSLv23SessionReq Probe
+Probe TCP TLSv1SessionReq 
q|\x16\x03\x01\x00j\x01\x00\x00f\x03\x01I\x8f\x16)\xa0_\xe2\xac\xe6\xfa\xea}$\xd4iH-\xa1^\x9ah\xa28}\xf5\x96\xe8\xc8\xde\x95T\x98\x00\x008\x00:\x009\x008\x005\x004\x003\x002\x00/\x00\x1b\x00\x1a\x00\x19\x00\x18\x00\x17\x00\x16\x00\x15\x00\x14\x00\x13\x00\x12\x00\x11\x00\n\x00\t\x00\x08\x00\x06\x00\x05\x00\x04\x00\x03\x00\x02\x00\x01\x02\x01\x00\x00\x04\x00#\x00\x00|
+
+rarity 3 
+ports 443,444,548,636,993,1241,1311,2000,4433,4444,5550,7210,7272,8009,8194,9001
+fallback SSLv23SessionReq
+
+#By default no matches, as we will fall back to the general matches in SSLv23SessionReq
+#match TLSv1 ServerHello:
+#match tlsv1 m|^\x16\x03\x01..\x02...\x03\x01|
+
+##############################NEXT PROBE##############################
+#This probe is redundant, as a SSLv3-only server will respond to the TLSv1 probe above as well
+#Note that a TLSv1-only client will not accept a SSLv3 Server Hello, and break the connection upon receiving the SSLv3 
Server Hello, but this is irrelevant for our purpose
+#Probe TCP SSLv3SessionReq 
q|\x16\x03\0\0S\x01\0\0O\x03\0?G\xd7\xf7\xba,\xee\xea\xb2`~\xf3\0\xfd\x82{\xb9\xd5\x96\xc8w\x9b\xe6\xc4\xdb<=\xdbo\xef\x10n\0\0(\0\x16\0\x13\0\x0a\0f\0\x05\0\x04\0e\0d\0c\0b\0a\0`\0\x15\0\x12\0\x09\0\x14\0\x11\0\x08\0\x06\0\x03\x01\0|
+
+#rarity 3
+#ports 443,444,548,636,993,1241,1311,2000,4433,4444,5550,7210,7272,8009,8194,9001
+#fallback SSLv23SessionReq
+
+#By default no matches, as we will fall back to the general matches in SSLv23SessionReq
+# Very generic; match SSLv3 Server Hello:
+#match sslv3 m|^\x16\x03\0..\x02...\x03\0
+
 # Apple Filing Protocol (AFP) over TCP on Mac OS X
-match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0\x01.\0\0\0\0\0.\0.\0.\0.\x80\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x05\x06AFPX03\x06AFP2\.2\x0eAFPVersion
 2\.1\x0eAFPVersion 2\.0\x0eAFPVersion 1\.1.\tDHCAST128|s p/Apple AFP/ i/name: $1; protocol 2.2; Mac OS X 10.1.*/
-match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0..\0\0\0\0\0.\0.\0.\0.\x83\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x06\x06AFP3\.1\x06AFPX03\x06AFP2\.2\x0eAFPVersion
 2\.1\x0eAFPVersion 2\.0\x0eAFPVersion 1\.1.\tDHCAST128|s p/Apple AFP/ i/name: $1; protocol 3.1; Mac OS X 10.2.*/
-match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0..\0\0\0\0\0.\0.\0.\0.\x83\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x03\x06AFP3\.1\x06AFPX03\x06AFP2\.2.\tDHCAST128|s
 p/Apple AFP/ i/name: $1; protocol 3.1; Mac OS X 10.3.*/
-match afp 
m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0.\0.\0..\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x04\x06AFP3\.2\x06AFP3\.1\x06AFPX03\x06AFP2\.2.\tDHCAST128|s
 p/Apple AFP/ i|name: $1; protocol 3.2; Max OS X 10.4/10.5|
-match afp 
m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0...\0..\xfa.([^\0\x01]+)[\0\x01].*\tMacintosh\x01\x06AFP3\.1.\tDHCAST128|s 
p/Apple Airport Extreme AFP/ i/name: $1; protocol 3.1/ d/WAP/
-match afp m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0...\0..\xfb.([^\0\x01]+)[\0\x01].*AirPort.*AFP3\.2|s p|Apple Airport 
Extreme/Time Capsule AFP| i/name: $1; protocol 3.2 WAP/
+#match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0\x01.\0\0\0\0\0.\0.\0.\0.\x80\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x05\x06AFPX03\x06AFP2\.2\x0eAFPVersion
 2\.1\x0eAFPVersion 2\.0\x0eAFPVersion 1\.1.\tDHCAST128|s p/Apple AFP/ i/name: $1; protocol 2.2; Mac OS X 10.1.*/
+#match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0..\0\0\0\0\0.\0.\0.\0.\x83\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x06\x06AFP3\.1\x06AFPX03\x06AFP2\.2\x0eAFPVersion
 2\.1\x0eAFPVersion 2\.0\x0eAFPVersion 1\.1.\tDHCAST128|s p/Apple AFP/ i/name: $1; protocol 3.1; Mac OS X 10.2.*/
+#match afp 
m|^\x01\x03\0\0\xff\xff\xecQ\0\0..\0\0\0\0\0.\0.\0.\0.\x83\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x03\x06AFP3\.1\x06AFPX03\x06AFP2\.2.\tDHCAST128|s
 p/Apple AFP/ i/name: $1; protocol 3.1; Mac OS X 10.3.*/
+#match afp 
m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0.\0.\0..\xfb.([^\0\x01]+)[\0\x01].*\tMacintosh\x04\x06AFP3\.2\x06AFP3\.1\x06AFPX03\x06AFP2\.2.\tDHCAST128|s
 p/Apple AFP/ i|name: $1; protocol 3.2; Max OS X 10.4/10.5|
+#match afp 
m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0...\0..\xfa.([^\0\x01]+)[\0\x01].*\tMacintosh\x01\x06AFP3\.1.\tDHCAST128|s 
p/Apple Airport Extreme AFP/ i/name: $1; protocol 3.1/ d/WAP/
+#match afp m|^\x01\x03\0\0....\0\0..\0\0\0\0\0.\0...\0..\xfb.([^\0\x01]+)[\0\x01].*AirPort.*AFP3\.2|s p|Apple Airport 
Extreme/Time Capsule AFP| i/name: $1; protocol 3.2 WAP/
 
-match login m|^\0\r\nlogin: \^W\^@\^@\^@\^| p/VxWorks logind/ o/VxWorks/
+#match login m|^\0\r\nlogin: \^W\^@\^@\^@\^| p/VxWorks logind/ o/VxWorks/
 
-match maxdb m|^.Rejected bad connect packet\0$|s p/SAP MaxDB/
+#match maxdb m|^.Rejected bad connect packet\0$|s p/SAP MaxDB/
 
 # OpenSSL/0.9.7aa
-match ssl m|^\x16\x03\0\0J\x02\0\0F\x03\0| p/OpenSSL/
+#match ssl m|^\x16\x03\0\0J\x02\0\0F\x03\0| p/OpenSSL/
 
 # Don't think these 2 are correct:
 #match ssl m|^\x16\x03\0\x04#\x02\0\0F\x03\0| p/Apache Tomcat SSL/
 #match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0| p/Apache mod_ssl/
 
 # Microsoft-IIS/5.0 - note that OpenSSL must go above this one because this is more general
-match ssl m|^\x16\x03\0..\x02\0\0F\x03\0|s p/Microsoft IIS SSL/ o/Windows/
+#match ssl m|^\x16\x03\0..\x02\0\0F\x03\0|s p/Microsoft IIS SSL/ o/Windows/
 # Novell Netware 6 Enterprise Web server 5.1 https
 # Novell Netware Ldap over SSL or enterprise web server 5.1 over SSL
-match ssl m|^\x16\x03\0\0:\x02\0\x006\x03\0| p/Novell Netware SSL/ o/NetWare/
-# Very generic:
-match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0|
+#match ssl m|^\x16\x03\0\0:\x02\0\x006\x03\0| p/Novell Netware SSL/ o/NetWare/
+# Very generic;
+#match sslv3 m|^\x16\x03\0\0\*\x02\0\0&\x03\0|
 # Cisco IDS 4.1 Appliance
-match ssl 
m|^\x16\x03\0\0\*\x02\0\0&\x03\0\xd10:\xbd\\\x8e\xe3\x15\x1c\x0fZ\xe4\x04\x87\x07\xc0\x82\xa9\xd4\x0e\x9c1LXk\xd1\xd2\x0b\x1a\xc6/p\0\0\n\0\x16\x03\0\x026\x0b\0\x022\0|
 p/Cisco IDS SSL/ d/firewall/
+#match ssl 
m|^\x16\x03\0\0\*\x02\0\0&\x03\0\xd10:\xbd\\\x8e\xe3\x15\x1c\x0fZ\xe4\x04\x87\x07\xc0\x82\xa9\xd4\x0e\x9c1LXk\xd1\xd2\x0b\x1a\xc6/p\0\0\n\0\x16\x03\0\x026\x0b\0\x022\0|
 p/Cisco IDS SSL/ d/firewall/
 # These Nessus match lines might be problematic:
-match ssl m|^\x15\x03\0\0\x02\x02\($| p/Nessus security scanner/ 
-match ssl m|^\x16\x03\x01\0J\x02\0\0F\x03\x01| p/Nessus security scanner/
+#A reply starting with 15 indicates an alert, in this stage most probably a handshake failure
+#match ssl m|^\x15\x03\0\0\x02\x02\($| p/Nessus security scanner/ 
+#match ssl m|^\x16\x03\x01\0J\x02\0\0F\x03\x01| p/Nessus security scanner/
 # PGP Corporation Keyserver Web Console 7.0 - custom Apache 1.3
 # PGP LDAPS Keyserver 8.X
-match ssl m|^\x16\x03\0\0\+\x02\0\0'\x03\0...\?|s p/PGP Corporation product SSL/
+#match ssl m|^\x16\x03\0\0\+\x02\0\0'\x03\0...\?|s p/PGP Corporation product SSL/
 # Unreal IRCd SSL
 # RemotelyAnywhere
-match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0\?|
+#match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0\?|
 # Tumbleweed SecureTransport 4.1.1 Transaction Manager Secure Port on Solaris
 # Dell Openmanage
-match ssl m|^\x15\x03[\x01\x00]\0\x02\x01\0$| p/multi-vendor SSL/
+#match ssl m|^\x15\x03[\x01\x00]\0\x02\x01\0$| p/multi-vendor SSL/
 # Probably Oracle https?
-match ssl m|^}\0\x02\0\0\0\0\0\0\0\0\0\0\0\0\0| p/Oracle https/
-match ssl m|^\x15\x03\0\0\x02\x02\(31666:error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared 
cipher:s3_srvr\.c:881:\n| p/Webmin SSL Control Panel/
-match ssl m|^20928:error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol:s23_srvr\.c:565:\n| 
p/qmail-pop3d behind stunnel/
+#match ssl m|^}\0\x02\0\0\0\0\0\0\0\0\0\0\0\0\0| p/Oracle https/
+#match ssl m|^\x15\x03\0\0\x02\x02\(31666:error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared 
cipher:s3_srvr\.c:881:\n| p/Webmin SSL Control Panel/
+#match ssl m|^20928:error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol:s23_srvr\.c:565:\n| 
p/qmail-pop3d behind stunnel/
 
-match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0B| p/Tor over SSL/
-match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03.*IOS-Self-Signed-Certificate|s p/Cisco IOS ssl/ d/router/
+#match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03\0B| p/Tor over SSL/
+#match ssl m|^\x16\x03\0\0\*\x02\0\0&\x03.*IOS-Self-Signed-Certificate|s p/Cisco IOS ssl/ d/router/
 
-match xtel m|^\x15Annuaire \xe9lectronique| p/xteld/ i/French/
+#match xtel m|^\x15Annuaire \xe9lectronique| p/xteld/ i/French/
 
-match tor m|^\x16\x03\0\0\*\x02\0\0&\x03\0.*T[oO][rR]1.*[\x00-\x20]([-\w_.]+) <identity>|s p/Tor node/ i/Node name: $1/
+#match tor m|^\x16\x03\0\0\*\x02\0\0&\x03\0.*T[oO][rR]1.*[\x00-\x20]([-\w_.]+) <identity>|s p/Tor node/ i/Node name: 
$1/
 
 # Sophos Message Router
-match ssl/sophos m|^\x16\x03\0.*Router\$([a-zA-Z0-9_-]+).*Sophos EM Certification Manager|s p/Sophos Message Router/ 
h/$1/
-match ssl/sophos m|^\x16\x03\0.*Sophos EM Certification Manager|s p/Sophos Message Router/
+#match ssl/sophos m|^\x16\x03\0.*Router\$([a-zA-Z0-9_-]+).*Sophos EM Certification Manager|s p/Sophos Message Router/ 
h/$1/
+#match ssl/sophos m|^\x16\x03\0.*Sophos EM Certification Manager|s p/Sophos Message Router/
 
 
 # SMB Negotiate Protocol


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org