Re: IPv6 fingerprint database imputation of missing values

[Alexandru Geana <alex () alegen net>]

Over the weekend I played around with some parameters and here are some
of my findings:

1) I tried to impute TC and TCP_WSCALE with linear regression, but the
accuracy from cross validation reported by liblinear drops to 55% - 60%.
I decided to leave them out. Now the accuracy estimates are 62% - 65%.
The accuracy value varies more now than before. I changed the scripts to
run cross validation multiple times and on the same model and the values
can differ as stated.

2) After choosing the imputation strategies per feature and a value for
the cost used by liblinear, I started fingerprinting various OSes, the
results of which are attached. As expected, the novelty factors have
increased and I adjusted the limit to 50. For percentages, the greatest
difference is for Linux 3.2 (tested on a Debian 7), going up from 82.7%
to 91.8%. A newer Linux 3.18 (tested on a Fedora 21) goes from 2.0% to
8.9%, though still for the wrong version of the kernel. For Windows 7,
the accuracy drops from 96% to 70%, but the novelty factor of the 2 top
classes (both for Windows 7) are "normalized" and are almost equal
(45 and 47 as opposed to 2 and 13). The most problematic was FreeBSD 10
which decreased quite a lot and was not the top match anymore. Even so,
the top match had a very high novelty factor (almost double). I decided
to change the code in FPEngine.cc, the classify function. My idea was
to check all perfect matches and if there are more than one, to verify
the novelty of each. If there is only one perfect match with a novelty
score lower than the threshold, then nmap reports that one, otherwise
the old behaviour is followed. I attached a small diff to give a better
idea. This helps with the stats for FreeBSD fingerprinting.

Best regards,
Alexandru Geana
alegen.net

Attachment: linux_3.2
Description:

Attachment: linux_3.18
Description:

Attachment: windows_7
Description:

Attachment: freebsd_10.1
Description:

Attachment: FPEngine.cc.diff
Description:

Attachment: signature.asc
Description: Digital signature

_______________________________________________
Sent through the dev mailing list
https://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/