Nmap Development mailing list archives
Re: Any guide for Adding match rules into nmap-service-probes?
From: ryan chou <jkryanchou () gmail com>
Date: Mon, 8 Jun 2015 14:55:36 +0800
Hi Dainnel Miller, I'm so appreciated for your reply. It seems a complicated process. Although I have read nmap-service-probes in details and known the rule-format and what every directive does, I am still curious about how this works. Is there any 101 tutorial for writing those match rules? How to reduce false positive and false negtive? Especically for those private scripts you mentioned, could you give me a brief introduction on its principle and what they mainly do? Could you offer me some suggestions if I want to customize the nmap-service-probes file ? Could any other developers participate in the process? If possible, how could we contribute to it by not only just submitting fingperints unrecognized by nmap on the pages you mentiond, but also writing match rules and testing its converage for imporving those match rules' accuracy. what should we prepare before start contributing? If not, could you give me the reasons why impossible? maintainces cost? or any other reasons? 2015-06-07 22:35 GMT+08:00 Daniel Miller <bonsaiviking () gmail com>:
Ryan, There is a set of scripts that streamlines the process of sorting and analyzing the hundreds of new service fingerprints that get submitted to the submitter page [1] every quarter. While the specific process and the scripts are private, the general process goes something like this: The fingerprint format is documented in the Nmap book and available online [2]. First, the fingerprint must be unwrapped, leaving a Probe name and the regex-escaped response which that probe elicited. In the most simple cases (including most "banner"-type services like FTP, SSH, and Telnet), the response is simply used directly as the match line, following the format described at [3]. Other times, some data may change in an insignificant way (like a timestamp or nonce) and it needs to be matched more generally, using regex atoms like "." or character classes. Choosing which probe response to use is a matter of optimizing some variables: 1. Choose a probe that is sent early, so that -sV doesn't take too long 2. Choose a probe that gets a unique response over one that is generic. This is why we have probes like FourOhFourRequest. 3. Choose a probe that may get fallback responses (Null or GetRequest) if there is some variability in the responses sent. The next step is to capture (as PCRE understands "capture") the portions of the response that represent interesting data that may change, like a hostname, version number, OS type, or other interesting info. These captures are then used in the version fields so that they can be presented to the user. The best way to understand this process (beyond reading the relevant chapter in the Nmap book [4]) is to read portions of the nmap-service-probes file. Dan [1] https://nmap.org/cgi-bin/submit.cgi?new-service [2] https://nmap.org/book/vscan-community.html#vscan-submit-prints [3] https://nmap.org/book/vscan-fileformat.html#vscan-db-match On Sat, Jun 6, 2015 at 7:24 PM, ryan chou <jkryanchou () gmail com> wrote:and i know the fingerprint-submit page, while i want to how the fingerprint-submitted be converted to match rule. any Guide or standard for this. as far as i know not all fingerprint which no match rules matched could be transform to match rule.: ) 2015-06-07 8:16 GMT+08:00 ryan chou <jkryanchou () gmail com>:Thanks Jacek, i Got it. i will resend this mail to other developers. :) 2015-06-06 23:56 GMT+08:00 Jacek Wielemborek <d33tah () gmail com>:W dniu 06.06.2015 o 17:14, ryan chou pisze:Hi Daniel Miller,Hello, You sent the message to Nmap developers list, so I'd assume that you would accept an answer from any other developer as well. Please find the answer below.May i ask you some questions on the process of adding match ruleintonmap-service-probes? Are there any guide or standard for it? I onlyfoundthe vs-fileformat instructions in nmap offical Guide. No charpterswereabout the process of writing regexes of rules and adding them into nmap-service-probes. Could you introduce it for us with some samples included the partof howto test it for improving the rules' coverage and reducing the rate offalsepositive. And Could it be a standard or Guide for more develpers toparticipate init to help improve the nmap?Generally, the best way to contribute is to report a fingerprint Nmap generates to https://nmap.org/submit/ . A sample fingerprint might look like this: nmap localhost -p 31337 -sV Starting Nmap 6.47SVN ( http://nmap.org ) at 2015-06-06 17:54 CEST Nmap scan report for localhost (127.0.0.1) Host is up (0.000045s latency). Other addresses for localhost (not scanned): ::1 PORT STATE SERVICE VERSION 31337/tcp open Elite? 1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service : SF-Port31337-TCP:V=6.47SVN%I=7%D=6/6%Time=557317B1%P=x86_64-unknown-linux- SF:gnu%r(NULL,2,"b\n"); Service detection performed. Please report any incorrect results at https://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 1.20 seconds The part you should submit is the following: SF-Port31337-TCP:V=6.47SVN%I=7%D=6/6%Time=557317B1%P=x86_64-unknown-linux- SF:gnu%r(NULL,2,"b\n"); (it might differ depending on the buffer you received) As for the nmap-service-probes modification, you will find documentation here: https://nmap.org/book/vscan-fileformat.html Please let us know if there is anything else you would like to know about. Cheers, d33tah_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Any guide for Adding match rules into nmap-service-probes? ryan chou (Jun 06)
- Re: Any guide for Adding match rules into nmap-service-probes? Jacek Wielemborek (Jun 06)
- Re: Any guide for Adding match rules into nmap-service-probes? ryan chou (Jun 06)
- Re: Any guide for Adding match rules into nmap-service-probes? ryan chou (Jun 06)
- Re: Any guide for Adding match rules into nmap-service-probes? Daniel Miller (Jun 07)
- Re: Any guide for Adding match rules into nmap-service-probes? ryan chou (Jun 07)
- Re: Any guide for Adding match rules into nmap-service-probes? ryan chou (Jun 06)
- Re: Any guide for Adding match rules into nmap-service-probes? Jacek Wielemborek (Jun 06)