Nmap Development mailing list archives
Re: PortBunny - FX and Fabs at 24C3
From: Brandon Enright <bmenrigh () ucsd edu>
Date: Sun, 13 Jan 2008 11:30:50 +0000
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A very thoughtful response in the face of such a negative presentation; thanks Fyodor. I'm constantly impressed by the integrity and comradeship of Nmap developers. My comments inline. On Sun, 13 Jan 2008 00:56:39 -0800 or thereabouts Fyodor <fyodor () insecure org> wrote:
On Sat, Jan 05, 2008 at 04:19:39PM +0100, kx wrote:Effectively, they find responsive hosts to acts as triggers for detecting packet loss. Dan Kaminsky was talking about this stuff at least as far back as 2005 with Scanrand, but I am sure earlier. He called them canaries as opposed to triggers. I watched the video, and I don't know what version of nmap they were testing against. I am not sure if they looked at David's code: http://seclists.org/nmap-dev/2007/q3/0433.htmlThanks for letting us know! Recurity has placed the code, slides, and presentation video up at http://www.recurity-labs.com/portbunny/portbunny.html . I watched the presentation and, to be honest, I'm not too impressed overall. Maybe I'd be more charitable if Fabs didn't attack Nmap throughout the whole thing. Oh wait, slide number 7 is titled "My hat is off to Fyodor!". Then he attacks Nmap for the remaining 59 slides :). Oh well, I'm sort of used to this after all the other presentations I've watched about amazing new scanners set to destroy Nmap. But Scanrand and UnicornScan haven't exactly eaten our lunch, and I don't think we need to start quivering in fear over PortBunny just yet :).
Unicornscan and Scanrand have their place but anyone claiming they should replace Nmap doesn't know what they're talking about. Unicornscan and Scanrand are niche port scanners tuned to do one or two things well. Nmap is the Swiss Army Knife of port scanners. Slide 10 of PortBunny says: "Bottom line: Do just one thing, but do it right.". That's great but Nmap does everything and it does it *very* well. But hey, if they think Nmap is the app to beat, we must be doing something right.
Here is the full description from the PortBunny web page: "PortBunny is a Linux-kernel-based port-scanner created by Recurity Labs. Its aim is to provide a reliable and fast TCP-SYN-port-scanner which performs sophisticated timing based on the use of so called "trigger"-packets. The port-scan is performed in 2 steps: First the scanner tries to find packets, to which the target responds ("triggers"). Second, the actual port-scan is performed. During the scan, the triggers, which were found in the first scanning-phase, are used to determine the optimal speed at which the target may be scanned." Clearly they consider "triggers" to be PortBunny's primary innovation. So I hate to break it to them that Nmap has done this for 3.5 years. Here is a snippet which was added to the Nmap Changelog (http://insecure.org/nmap/changelog.html) for the 3.70 release (August 2004): o Nmap now has a "port scan ping" system. If it has received at least one response from any port on the host, but has not received responses lately (usually due to filtering), Nmap will "ping" that known-good port occasionally to detect latency, packet drop rate, etc. That sounds a lot like their "triggers", doesn't it? It is true that they send more triggers which does greatly improve speed, but it can lead to accidentally syn-flooding the open port you use as a trigger. (this is not a problem if you find a closed port, but this is only useful for very heavily filtered hosts anyway).
"Triggers" are a good idea, even if Nmap beat them to it. We may be able to better borrow a few ideas and better utilize our port ping system.
In Slide 35, Fabs does a comparison scan with "66535 ports, mostly filtered, Internet". I'm sure he means 65,535 ports, but that is just a typo. Anyway, Nmap (no version number or command line given) takes 12 minutes, 18 seconds. PortBunny took 15 seconds. Of course, the audience cheered wildly even though (IMHO) this is a sign that PortBunny is WAY too aggressive. That is an average of 4,369 probes per second (not counting all of their trigger probes). Nmap can scan that fast too if you remove congestion control and toss out the idea of accuracy.
Unicornscan would have taken about 8 seconds. Anyone actually looking an the scan results of these scans though would see right away that faster doesn't mean better. Too bad they didn't include results.
The presentation is filled with scan times, but none of the slides address accuracy. It appears that they don't do any sort of retransmission, so if their probe to a port is dropped, they just count it as filtered and move on. Nmap by default retransmits to each port at least once. But if they are so concerned with speed, they can turn that off. Also, they could try options like -T5 to make Nmap more aggressive. Admittedly Nmap has far more timing-related options than PortBunny has total options, but learning the Nmap timing options is still much easier than writing your own port scanner. Too bad they didn't spend the time improving what they see as subpar algorithms in Nmap's approach to dealing with heavily filtered hosts.
Patches would have been nice. They wanted to reinvent the wheel though (slide 2).
Their slide 50 describes "serious bugs" in Nmap, but the algorithm they spotted is actually a critical feature for UDP scans (and can be useful in rare TCP cases too). Maybe if their scanner actually handled UDP, they would understand this.
They fail to mention that not doing this will make the scan less accurate. Slide 50 is a good example of how desperate they are criticize.
Anyway, I try not to just get mad when I see presentations like this, and to actually try to think of ways to improve Nmap. Here are some things we might look at: o Whether we should do the port scan pings more frequently. Fabs makes fun of us for only doing one every five seconds, but that slow rate was chosen for a very good reason: we don't want to SYN flood the ports.
I think we should increase the rate to at least 1/s for the faster timing options.
Remember that many old BSD machines and crappy devices can be SYN flooded with as few as 8 packets.
Maybe a tunable parameter? I don' have a problem with --yet-another-timing-option, this is one on the reason why Nmap is so flexible.
Also, if the machine you are running Nmap from is firewalled such that it isn't sending RST in response to the SYN/ACK returned from the port scan ping (trigger) packets, a SYN flood is even more likely. Still, we might be able to make it somewhat more aggressive. We could also consider having Nmap generate its own RST in response to SYN/ACK packets, though it would be best if we only do that if the host isn't doing this itself. And that detection may be a bit of a pain.
There are a bunch of reasons why it is a very bad idea to run Nmap on a firewalled machine. We should probably remind people of this more often rather than worrying about sending RSTs on our own.
o Fabs claims that our global congestion control (which works on the set of hosts were are scanning, as opposed to single targets) may be dragged down undesirably when only 1 target is dropping many packets but the rest of the hosts are very fast and response. We should investigate and think more about how we want Nmap to behave in such situations. Portbunny attacks the Nmap algorithms for scanning hosts in parallel, but it sort of punts the issue by only scanning one host at a time by default. We could consider loosening the global congestion controls, removing them, or offering an option to remove them. But that would require more data first.
This is the one argument that I agree with; but not for the same reasons they cite. Some of this has improved with David's work moving massping() to ultrascan() but the problem hasn't completely gone away. My experience is that Nmap is very good at a few number of ports across many hosts or many ports across a few hosts. In fact, I too "solved" this problem by only scanning a single host at a time with Nmap. To still scan in parallel, I run many instances of Nmap at once. I call it poor man's threading. If you have the hardware to do it and you've appropriately tuned your kernel, it works *very* well. Another reason I do this is that Nmap used to (may have been fixed) time out a whole scan group / host group when a single host timed out. I'd have a single "bad" host kill the results for a whole group of 32 or more. I suspect the original reason for global congestion control was that the network was as much the bottleneck as a single host. I think slides 53 and 54 are complete bullshit now though. The only case where the network is a shared bottleneck is wireless networks and *really old* hardware based networks. Modern switching and routing gear can handle millions of pps, far more than a single scanning machine can generate. The bottleneck is how quickly the target host can respond.
o Fabs declares that "the kernel is a good place for a port scanner", and indeed PortBunny is a Linux-only kernel module. They suggest that you use a dedicated box and not run other Internet applications such as web browsers at the same time.
This is a terrible idea. I think they did it in the kernel because they wanted to and went looking for excuses why it was a good idea later. If speed is all you're looking for, the Unicornscan guys sure seem to be doing well with their user-land distribute TCP/IP stack. There is absolutely no good reason to stuff a portscanner in the kernel. With this design, what they've made is a packet generator that happens to tell you what ports are open.
I don't think we'll copy this load-the-port-scanner-into-the-kernel approach :). They complain that Nmap is too bloated (do you guys really not like or use OS detection, version detection, etc.?), but at least Nmap doesn't bloat your kernel.
Nmap is great and it continues to get development and new features because people like it and want it to do more. I say as long as we keep the quality up, new features are more than welcome.
Anyway, those are my initial reactions. While it is a shame that they took such an antagonistic we're-better-than-Nmap approach, it is always nice to see new open source port scanners and research! I hope they continue working on it (but tone down their presentations).
I 100% agree. They have some good ideas and have clearly thought a lot about port scanning. I'd like to see more research more numbers (including accuracy info) out of them.
Another thing I'll give them is that they do already have shirts for sale. Someday we need to make bad-ass Nmap shirts :). Cheers, Fyodor
To summarize things I think we should look at: * Look into increasing the port-ping rate and possibly even make it tunable. I'd like an option like --heartbeat-delay <ms> * Look into reducing the effects of global congestion control, moving to more of a per-host timing setup. Maybe allow this to be turned on or off with an option like --timing-per-host I'll volunteer to provide any host/group timing information if someone wants to get "real world" data from a large network. We now have a 100% dedicated Nmap box that does nothing but Nmap scan like crazy all day long. It should be easier for me to get consistent results now than it was when David was migrating massping(). Brandon -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) iD8DBQFHifZyqaGPzAsl94IRAth3AJ9bgJX5Hh+p0DeiW0D07MdxTHlfswCgv+su fv34lGR4OuZ6CoWpCQxsO4w= =+NCq -----END PGP SIGNATURE----- _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- PortBunny - FX and Fabs at 24C3 kx (Jan 05)
- Re: PortBunny - FX and Fabs at 24C3 Fyodor (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Brandon Enright (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 doug (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 bensonk (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Tyler Reguly (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Fyodor (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Tyler Reguly (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Tyler Reguly (Jan 14)
- Re: PortBunny - FX and Fabs at 24C3 Brandon Enright (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Fyodor (Jan 13)
- Re: PortBunny - FX and Fabs at 24C3 Martin Mačok (Jan 15)
- <Possible follow-ups>
- Re: PortBunny - FX and Fabs at 24C3 Robert E. Lee (Jan 24)