Re: Ideas for rate limit detection?

[David Fifield <david () bamsoftware com>]

On Thu, Jan 08, 2009 at 11:40:49PM +0000, Brandon Enright wrote:
> On Fri, 09 Jan 2009 00:54:50 +0200
> "ithilgore.ryu.L () gmail com" <ithilgore.ryu.l () gmail com> wrote:
>
> ...snip...
>
> > In addition, we could take a look into what kind of traffic is usually
> > rate-limited (maybe by observing best-practices/patterns on firewall
> > policies), so that we could mainly focus on it and thus extract more
> > accurate results. For example, if ICMP echo replies are usually
> > rate-limited, then Nmap should be more on the lookout for rate-limit
> > behaviour rather than network congestion, on this particular kind of
> > traffic.
>
> I don't categorically disagree with your above point but I'll add that
> at high scan speeds almost everything is rate-limited. For example,
> Windows seems to have a 4000-5000 RST-per-second rate limit. I suppose
> one could argue that if you're scanning at 3000+ pps who cares that
> you /could/ be doing 4000+.  The trouble used to be that Nmap would
> increase the scan delay to 5ms and the rate would go from 4000+ to
> 200.  David has fixed that.

I should make clear that come up with an algorithm that nicely solved
this particular problem, but I couldn't make it work as well as the
granular scan delay in all situations, so it was not included in the
recent nmap-perf merge.

> OS X has a 150-250 RST-per-second limit.  There is so much variance
> between 4000-5000 and 150-250 that I think it is hard to make a simple
> rate-limit/congestion discriminator.  Add to that the wildly varying
> firewall configurations out there and the task is near impossible.

Yes, the OS X rate limiter is weird. It's supposed to be 250 packets per
second, which you can verify with "sysctl net.inet.icmp.icmplim". But I
could only push it to about 125 packets per second with no drops. Even
at 130 packets per second, you get messages in the system log:
kernel[0]: Limiting closed port RST response from 260 to 250 packets per second
kernel[0]: Limiting closed port RST response from 259 to 250 packets per second
So there seems to be some sort of measurement error. At
http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive1#rate-scatter
I found that at 160 packets per second you start to miss ports.

OS X also limits ICMP destination unreachables to 250 per second by
default. ICMPs and RSTs are controlled by the same sysctl:
net.inet.icmp.icmplim. Nmap has a heuristic where it increases the scan
delay straight to 50 ms rather than 5 ms during a UDP scan, anticipating
a Linux-style limit of 1 per second. That heuristic falls down hard
against OS X, because a delay of 50 ms means a rate of 20 packets per
second, about 8% of the nominal and 16% of the practical rate limit.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org