fingerprint database

[rieger () dest-unreach org]

Hi N-Mappers!

For me fingerprinting is nmap's most interesting feature. But recently I got
a "Mac OS" response for a system that turned out to be some HP-UX box; this
inspired some doubts about the public fingerprint collection. And there are
more reasons to doubt its integrity:

A simple "grep -i linux nmap-os-fingerprints" results in something like this
(sorted by kernel version):

Fingerprint Linux 1.0.9
Fingerprint Linux 1.2.8 - 1.2.13
Fingerprint Linux 1.2.13
Fingerprint Linux 2.0.0
Fingerprint Linux 2.0.27 - 2.0.30
Fingerprint Linux 2.0.32-34
Fingerprint Linux 2.0.32-34
Fingerprint Cobalt Linux 4.0 (Fargo) Kernel 2.0.34C52_SK on MIPS or TEAMInternet Series 100 WebSense
Fingerprint Linux 2.0.34-38
Fingerprint Linux 2.0.35 (S.u.S.E. Linux 5.3 (i386)
Fingerprint Linux 2.1.19 - 2.2.17
Fingerprint Linux 2.1.24 PowerPC
Fingerprint Linux 2.1.76
Fingerprint Linux Kernel 2.1.88
Fingerprint Linux 2.1.91 - 2.1.103
Fingerprint Linux 2.2.5 - 2.2.13 SMP
Fingerprint Linux 2.2.12 - 2.2.19
Fingerprint Linux kernel 2.2.13
Fingerprint Linux 2.2.14
Fingerprint Linux 2.2.19
Fingerprint Linux 2.2.19 on a DEC Alpha
Fingerprint Linux 2.3.12
Fingerprint Linux 2.3.28-33
Fingerprint Linux 2.3.47 - 2.3.99-pre2 x86
Fingerprint Linux 2.3.49 x86
Fingerprint Linux Kernel 2.4.0-test5
Fingerprint Linux Kernel 2.4.0 - 2.4.5 (X86)

This looks a little chaotic with its sporadic hardware or distribution
dependence, overlapping version ranges, and ambiguities.

I waited for you gurus to clean this mess up, but it seems you waited for me
to do it ;-) so here I go:
A little research on the IP stack properties that fingerprinting checks
brought up some interesting points: window size, tcp options, and PMTU
discovery. Why interesting? Because many operating systems provide system wide
and/or per socket options to tune these values! While this fact seems to be
known (e.g., http://razor.bindview.com/publish/papers/tcpseq.html#conclude last
paragraph), the fingerprints database does not reflect it.

So, let's do some magic:
I found an opportunity to update me old Linux 2.2.10 kernel to 2.2.19 without
rebooting!
Here is the trick:
(you need a software that allows you to set the SO_RCVBUF and IP_MTU_DISCOVER
sockopts; you might checkout the beta version of socat at
http://www.dest-unreach.org/socat/)

xterm1# echo 0 >/proc/sys/net/ipv4/tcp_window_scaling
xterm1# echo 0 >/proc/sys/net/ipv4/tcp_timestamps
xterm2$ socat - tcp-l:10000,ip-mtu-discover=1,rcvbuf384
xterm1# nmap -sS -p 10000,10001 -O loopback

This rewards my investigations with a "Linux 2.2.19" guess.
But, even better: some linux parameters can add another cpu to my cheap
motherboard:

xterm1# echo 0 >/proc/sys/net/ipv4/tcp_window_scaling
xterm1# echo 0 >/proc/sys/net/ipv4/tcp_timestamps
xterm2$ socat - tcp-l:10000,ip-mtu-discover=1,rcvbuf`0
xterm1# nmap -sS -p 10000,10001 -O loopback

introduces my new multiprocessor board: "Linux 2.2.5 - 2.2.13 SMP"!

ok, seriously again:
What looked to Fyodor as version dependend properties turns out to be mostly
just some sysctl or socket parameters. And the same is true for other operating
systems. So, is this the end of fingerprinting?
No, still nmap's fingerprinting is strong enough to distinguish between major
releases of linux and other operating systems.

What can we do about this? One possibility is a more general approach to avoid
ambiguities: just identify which operating system versions and releases can be
distinguished, and define a "general" fingerprint database entry for each of
these, "or'ing" all possible parameter variations in one fingerprint.
A guess result might then look something like
"Linux 2.2 or 2.3.1-2.3.49"

But this would drop a lot of information that Fyodors fingerprinting method
exploits. Therefore I think it might be better to create fingerprint database
entries that distinguish not only between releases, but also print out the
values of the relevant OS and socket parameters.
A guess would then look something like
"Linux 2.2 or 2.3.1-2.3.49 with so_rcvbufe535 (default) ip_no_pmtu_disc=0 tcp_window_scaling=0 tcp_timestamps=1"

If we survive this multiline monster, we could check for default parameter
values of vanilla linux kernels and common linux distributions, so a
fingerprint name might look like
"Linux 2.2 or 2.3.1-2.3.49 with so_rcvbufe535 (default) ip_no_pmtu_disc=0 tcp_window_scaling=0 tcp_timestamps=1, 
BlueCap Linux 6.1-6.2 default, Flash Linux 2.2 default"
(Note that these versions are NOT verified, they are just examples)

While the parameter settings that we can determine this way are for performance
tuning rather than security, it might still give us hints about the quality of
the system administration: with some probability we may assume that "out of the
box" installations will be less patched and less secured than hosts with kernel
parameters tuned.

These are the parameters I found important for nmap's fingerprinting on
Linux 2.2:

system parameter        socket option           fingerprint field
----------------        -------------           -----------------
ipv4/ip_no_pmtu_disc    IP_MTU_DISCOVER         T1-T8(DF=...)
ipv4/ip_window_scaling  -                       T1-T3(Ops=*W*)
ipv4/ip_timestamps      -                       T1-T3(Ops=*T*)
core/rmem_default       SO_RCVBUF               T1-T3(W=...)

I have created fingerprints for Linux 2.2.10 with combinations of the above
parameters and suggest they replace the legacy linux fingerprints after a
period of testing.

Would be nice if more of us could help Fyodor to improve the nmap fingerprints
database by finding more sensitive parameters and analyzing their dependencies
on other linux versions and other operating systems.

greets
Gerhard Rieger

--------------------------------------------------
For help using this (nmap-hackers) mailing list, send a blank email to 
nmap-hackers-help () insecure org . List run by ezmlm-idx (www.ezmlm.org).