Nmap Development mailing list archives
Re: Feature: per-target port specification (with patch!)
From: Jan Gocník <gocnik () dcit cz>
Date: Tue, 2 Apr 2019 23:53:31 +0200
Hey Dan, thanks for the reply! It's a shame that I didn't find the GitHub issue you link to before implementing this, as it does raise a lot of valid concerns. First, let me say that the company I work for wants this feature, so even if it doesn't end up in upstream, I will try to keep it at least as a fork - as I'll have to maintain it internally anyway, I wanted to share with the community, in case others have a need for it as well. I will go through all the things you mentioned (most of the compatibility with other options should be taken care of, but memory leaks are a problem), fix up the code, look at maybe getting the memory footprint lower, and try to come up with some stronger numbers and rationale. Jan From: "Daniel Miller" <bonsaiviking () gmail com> To: "Jan Gocník" <gocnik () dcit cz> Cc: "Nmap-dev" <dev () nmap org> Date: 02.04.2019 21:21 Subject: Re: Feature: per-target port specification (with patch!) Sent by: "dev" <dev-bounces () nmap org> Some initial notes from building and testing this: ./nmap scanme.nmap.org^22-80 -d Starting Nmap 7.70SVN ( https://nmap.org ) at 2019-04-02 18:37 UTC PORTS: Using top 1000 ports found open (TCP:1000, UDP:0, SCTP:0) --------------- Timing report --------------- hostgroups: min 1, max 100000 rtt-timeouts: init 1000, min 100, max 10000 max-scan-delay: TCP 1000, UDP 1000, SCTP 1000 parallelism: min 0, max 0 max-retries: 10, host-timeout: 0 min-rate: 0, max-rate: 0 --------------------------------------------- Initiating Ping Scan at 18:37 Scanning scanme.nmap.org (45.33.32.156) [max 2 ports] Completed Ping Scan at 18:37, 0.09s elapsed (1 total hosts) Overall sending rates: 24.44 packets / s. mass_rdns: Using DNS server X.X.X.X Initiating Parallel DNS resolution of 1 host. at 18:37 mass_rdns: 0.12s 0/1 [#: 3, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1] Completed Parallel DNS resolution of 1 host. at 18:37, 0.06s elapsed DNS resolution of 1 IPs took 0.15s. Mode: Async [#: 3, OK: 1, NX: 0, DR: 0, SF: 0, TR: 1, CN: 0] Initiating Connect Scan at 18:37 Scanning scanme.nmap.org (45.33.32.156) [max 1059 ports] Discovered open port 80/tcp Discovered open port 22/tcp Discovered open port 9929/tcp Discovered open port 31337/tcp nmap: portlist.cc:688: void PortList::mapPort(u16*, u8*) const: Assertion `mapped_portno < port_list_count[mapped_protocol]' failed. Aborted (core dumped) Valgrind identified some memory leaks. These were the ones that are definitely from this patch: Portlist::setIdStr(), idstr; PortList::mergeHostSpecificPorts(), new_port_map and new_port_map_rev; There was also a discrepancy between the "Scanning X [max N ports]" and "Completed Connect Scan at X (0 ports max)", which valgrind says is due to an uninitialized value, probably GroupScanStats::totalprobes. Of course, as you noted before, the results are not sorted by port number, which we would have to do to ensure predictable output that can be compared with previous scans. IPv6 addresses, CIDR, and IPv4 octet ranges seem to work fine with this. With just my one test scan of localhost and scanme.nmap.org with one unique port each and one port in common, the patched code allocated an additional 383KB of heap memory. I do not know how this would scale up or down, and I haven't compared it with a scan that should behave the same in both cases (for instance, "nmap 192.168.1.0/24". My primary concern remains that repeated use of this feature will result in missing new services and dropping existing ones if they are missed in just one scan. This is more about use case than about the code, though, so I will defer to users on that. Dan On Tue, Apr 2, 2019 at 1:30 PM Daniel Miller <bonsaiviking () gmail com> wrote: Jan, Thanks for this contribution. We've had many requests for this type of feature in the past, but have elected not to include it for a variety of reasons. There is an open discussion on our issue tracker that lays out some of the challenges in correctly implementing such a feature: http://issues.nmap.org/1217 It looks like your patch has tried to handle some of these situations, for example the "Ports scanned" output for Grepable output (and maybe XML, but it didn't look complete at first glance). If we are to do an actual code review and include this new feature, we would have to look for a complete solution that can handle the following situations: * The "Not shown: X ports" output for Normal output. * Properly formed XML output, with changes to the DTD and a "xmloutputversion" number increase. * Combination of this feature with existing --top-ports/port-ratio and -p options * Combination of this feature with CIDR subnetting and IPv4 octet ranges * Use of this feature along with advanced features like -O --traceroute and -sV Have you done any measurement of scans before and after adding this feature to determine the actual impact on scan times and bandwidth? Do you have a bandwidth target for your scans that Nmap is exceeding right now, and by how much? What does a typical nmap command line look like, and what performance options have you already tried? I look forward to hearing more about this from you and our other devs and users. Dan On Tue, Apr 2, 2019 at 8:07 AM Jan Gocník <gocnik () dcit cz> wrote: Hey, I would like to propose a feature enabling specifying ports for each target separately. Rationale: It often happens that we already have an nmap scan of 200 machines, and we want to do a service scan on those same machines. Usually that forces us to scan the whole network for all the ports that appeared at least once. That is a big waste of time and bandwidth. What we want to have is essentially a rescan-like feature, that would rescan just ports that were found to be open before. User experience: Everywhere where you could specify a target (-iL file, command line) you can supply a "target^ports". It works with all the nmap magic ranges, so "192.168.1.1-255^22-60" works. The common ports (supplied with -p) are scanned on all targets. Implementation details: I tried to keep it so that if you don't use any "^" in the targets, the code path should remain largely the same, so there should be no regressions. However, I had to do some tuning in functions that expected they can just get the number of probes by multiplying common ports by targets. There's a small issue, in that the results of the scan are not sorted properly, as the target-specific ports get scanned last. Usage example: ===paste start=== $ nmap -v -Pn -n -p22 "165.227.141.119^80,443" "40.113.73.59^8080" Starting Nmap 7.70SVN ( https://nmap.org ) at 2019-04-01 19:46 CEST Initiating SYN Stealth Scan at 19:46 Scanning 2 hosts [max 3 ports/host] Discovered open port 22/tcp Discovered open port 80/tcp Discovered open port 443/tcp Discovered open port 22/tcp Completed SYN Stealth Scan at 19:46, 1.45s elapsed (1626388576 total ports max) Nmap scan report for 165.227.141.119 Host is up (0.0090s latency). PORT STATE SERVICE 22/tcp open ssh 80/tcp open http 443/tcp open https Nmap scan report for 40.113.73.59 Host is up (0.038s latency). PORT STATE SERVICE 22/tcp open ssh 8080/tcp filtered http-proxy Read data files from: /home/gocnik/nmap Nmap done: 2 IP addresses (2 hosts up) scanned in 1.52 seconds Raw packets sent: 6 (264B) | Rcvd: 4 (176B) ===paste end=== If done the usual way: $ nmap -v -Pn -n -p22,80,443,8080 165.227.141.119 40.113.73.59 [...] Raw packets sent: 10 (440B) | Rcvd: 6 (260B) The patch is against svn trunk at this moment (revision 37608). Looking forward to all comments! JaGoTu P.S.: Sorry if you recieve this e-mail twice, but the previous one apparently got caught in a moderation queue or something, as it doesn't show on seclists.org _______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/ _______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Feature: per-target port specification (with patch!) Jan Gocník (Apr 02)
- Re: Feature: per-target port specification (with patch!) Daniel Miller (Apr 02)
- Re: Feature: per-target port specification (with patch!) Daniel Miller (Apr 02)
- Re: Feature: per-target port specification (with patch!) Jan Gocník (Apr 02)
- Message not available
- Re: Feature: per-target port specification (with patch!) Jan Gocník (Apr 08)
- Re: Feature: per-target port specification (with patch!) Robin Wood (Apr 08)
- Re: Feature: per-target port specification (with patch!) Jan Gocník (Apr 08)
- Re: Feature: per-target port specification (with patch!) Robin Wood (Apr 08)
- Re: Feature: per-target port specification (with patch!) Daniel Miller (Apr 02)
- Re: Feature: per-target port specification (with patch!) Daniel Miller (Apr 02)
- Message not available
- Message not available
- Re: Feature: per-target port specification (with patch!) Jan Gocník (May 03)