Re: PortBunny - FX and Fabs at 24C3

[Fyodor <fyodor () insecure org>]

On Sat, Jan 05, 2008 at 04:19:39PM +0100, kx wrote:
> Effectively, they find responsive hosts to acts as triggers for
> detecting packet loss. Dan Kaminsky was talking about this stuff at
> least as far back as 2005 with Scanrand, but I am sure earlier. He
> called them canaries as opposed to triggers.
>
> I watched the video, and I don't know what version of nmap they were
> testing against. I am not sure if they looked at David's code:
> http://seclists.org/nmap-dev/2007/q3/0433.html

Thanks for letting us know!  Recurity has placed the code, slides, and
presentation video up at
http://www.recurity-labs.com/portbunny/portbunny.html .

I watched the presentation and, to be honest, I'm not too impressed
overall.  Maybe I'd be more charitable if Fabs didn't attack Nmap
throughout the whole thing.  Oh wait, slide number 7 is titled "My hat
is off to Fyodor!".  Then he attacks Nmap for the remaining 59 slides
:).  Oh well, I'm sort of used to this after all the other
presentations I've watched about amazing new scanners set to destroy
Nmap.  But Scanrand and UnicornScan haven't exactly eaten our lunch,
and I don't think we need to start quivering in fear over PortBunny
just yet :).

Here is the full description from the PortBunny web page:
 
  "PortBunny is a Linux-kernel-based port-scanner created by Recurity
   Labs. Its aim is to provide a reliable and fast
   TCP-SYN-port-scanner which performs sophisticated timing based on
   the use of so called "trigger"-packets. The port-scan is performed
   in 2 steps: First the scanner tries to find packets, to which the
   target responds ("triggers"). Second, the actual port-scan is
   performed. During the scan, the triggers, which were found in the
   first scanning-phase, are used to determine the optimal speed at
   which the target may be scanned."

Clearly they consider "triggers" to be PortBunny's primary innovation.
So I hate to break it to them that Nmap has done this for 3.5 years.
Here is a snippet which was added to the Nmap Changelog
(http://insecure.org/nmap/changelog.html) for the 3.70 release (August
2004):

o Nmap now has a "port scan ping" system. If it has received at least
  one response from any port on the host, but has not received
  responses lately (usually due to filtering), Nmap will "ping" that
  known-good port occasionally to detect latency, packet drop rate,
  etc. 

That sounds a lot like their "triggers", doesn't it?  It is true that
they send more triggers which does greatly improve speed, but it can
lead to accidentally syn-flooding the open port you use as a trigger.
(this is not a problem if you find a closed port, but this is only
useful for very heavily filtered hosts anyway).

In Slide 35, Fabs does a comparison scan with "66535 ports, mostly
filtered, Internet".  I'm sure he means 65,535 ports, but that is just
a typo.  Anyway, Nmap (no version number or command line given) takes
12 minutes, 18 seconds.  PortBunny took 15 seconds.  Of course, the
audience cheered wildly even though (IMHO) this is a sign that
PortBunny is WAY too aggressive.  That is an average of 4,369 probes
per second (not counting all of their trigger probes).  Nmap can scan
that fast too if you remove congestion control and toss out the idea
of accuracy.

The presentation is filled with scan times, but none of the slides
address accuracy.  It appears that they don't do any sort of
retransmission, so if their probe to a port is dropped, they just
count it as filtered and move on.  Nmap by default retransmits to each
port at least once.  But if they are so concerned with speed, they can
turn that off.  Also, they could try options like -T5 to make Nmap
more aggressive.  Admittedly Nmap has far more timing-related options
than PortBunny has total options, but learning the Nmap timing options
is still much easier than writing your own port scanner.  Too bad they
didn't spend the time improving what they see as subpar algorithms in
Nmap's approach to dealing with heavily filtered hosts.

Their slide 50 describes "serious bugs" in Nmap, but the algorithm
they spotted is actually a critical feature for UDP scans (and can be
useful in rare TCP cases too).  Maybe if their scanner actually
handled UDP, they would understand this.

Anyway, I try not to just get mad when I see presentations like this,
and to actually try to think of ways to improve Nmap.  Here are some
things we might look at:

o Whether we should do the port scan pings more frequently.  Fabs
  makes fun of us for only doing one every five seconds, but that slow
  rate was chosen for a very good reason: we don't want to SYN flood
  the ports.  Remember that many old BSD machines and crappy devices
  can be SYN flooded with as few as 8 packets.  Also, if the machine
  you are running Nmap from is firewalled such that it isn't sending
  RST in response to the SYN/ACK returned from the port scan ping
  (trigger) packets, a SYN flood is even more likely.  Still, we might
  be able to make it somewhat more aggressive.  We could also consider
  having Nmap generate its own RST in response to SYN/ACK packets,
  though it would be best if we only do that if the host isn't doing
  this itself.  And that detection may be a bit of a pain.

o Fabs claims that our global congestion control (which works on the
  set of hosts were are scanning, as opposed to single targets) may be
  dragged down undesirably when only 1 target is dropping many packets
  but the rest of the hosts are very fast and response.  We should
  investigate and think more about how we want Nmap to behave in such
  situations.  Portbunny attacks the Nmap algorithms for scanning
  hosts in parallel, but it sort of punts the issue by only scanning
  one host at a time by default.  We could consider loosening the
  global congestion controls, removing them, or offering an option to
  remove them.  But that would require more data first.

o Fabs declares that "the kernel is a good place for a port scanner",
  and indeed PortBunny is a Linux-only kernel module.  They suggest
  that you use a dedicated box and not run other Internet applications
  such as web browsers at the same time.  I don't think we'll copy
  this load-the-port-scanner-into-the-kernel approach :).  They
  complain that Nmap is too bloated (do you guys really not like or
  use OS detection, version detection, etc.?), but at least Nmap
  doesn't bloat your kernel.

Anyway, those are my initial reactions.  While it is a shame that they
took such an antagonistic we're-better-than-Nmap approach, it is
always nice to see new open source port scanners and research!  I hope
they continue working on it (but tone down their presentations).

Another thing I'll give them is that they do already have shirts for
sale.  Someday we need to make bad-ass Nmap shirts :).

Cheers,
Fyodor



_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org