TCP congestion control and large router buffers

[Jim Gettys <jg () freedesktop org>]

As I'm attempting to lay out in my posts, there are are a plethora of 
problems, end-to-end in the network.  Would that there was only one problem.

Excessive, unmanaged buffers afflict the user's OS's (Windows, Mac and 
Linux alike), particularly on recent hardware. Home routers and the 
broadband connections (as shown by netalyzr) all have problems.

The bottleneck may be anywhere in the path; with the (sometimes) 
exception of Windows XP, all edge equipment now routinely congests the 
edge.

Multiple seconds of latencies, in both directions, are dismaying 
commonplace.

Retail operators have had a hidden major support problem: how many of 
the "bad service" calls have been due to the problem? It tends to be 
transient in behavior, and I've chased the problem personally at least 5 
times in the last 3 years.  I've placed service calls I now believe 
likely due to bufferbloat.  I've caught problems with crash dumps being 
uploaded to the net; backup and downloads can all cause trouble.

Courtesy of the Netalyzr team, I've been able to post color versions of 
their results on 
http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/; 
they first reported results at the NANOG meeting last summer.

Disentangling broadband data from home router and operating system 
bufferbloat is difficult; I've found bufferbloat is present in all of 
them.  The power of two bufferbloat sizes are almost certainly all 
broadband gear (since the OS buffer sizes are quantised in packets, not 
bytes).

In the downstream direction, one of the possible causes is be failure to 
run any AQM in the broadband head-ends; this is certainly also the case 
in home routers.

Also, as outlined in: Characterizing Residential Broadband Networks
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.6825&rep=rep1&type=pdf 
we have good reason to believe this is taking place.

Since I became aware that bufferbloat might have become a generic 
problem last summer from anecdotal data and personal experiments, I've 
probed networks wherever I've travelled.  Some of what I've seen was 
clearly broadband bufferbloat; but more disturbingly, I've also seen 
other evidence further into several of the networks I've probed (from 
hotels *not* using broadband for their service), further confirming the 
initial anecdotal data I was given that queue management is far from 
universal (and essentially unheard of in the home).

If the idea that the buffers have destroyed congestion avoidance doesn't 
scare you, I don't know what will.

It's a major problem.
                        Best Regards,
                                - Jim Gettys