nanog mailing list archives
RE: Extreme congestion (was Re: inter-domain link recovery)
From: <michael.dillon () bt com>
Date: Thu, 16 Aug 2007 23:20:19 +0100
The TCPs don't slow down. They use the bandwidth you have made available instead. in your words, "the traffic on the new circuit is suddenly greater than 100% of the old one".
Exactly! To be honest, I first encountered this when Avi Freedman upgraded one of his upstream connections from T1 to DS3 and either Avi, or one of his employees mentioned this on inet-access or nanog. So I did a bit of digging and discovered that other people had noticed that TCP traffic tends to be fractal (or multi-fractal) in nature. That means that the peaks which cause this effect are hard to get rid of entirely.
To his surprise, with the standing load > 95% and experiencing 20% loss at 311 MBPS, doubling the rate to 622 MBPS resulted in links with a standing load > 90% and 4% loss. He still needed more bandwidth. After we walked offstage, I explained TCP to him...
That is something that an awful lot of operations and capacity planning people do not understand. They still think in terms of pipes with TCP flavoured water flowing in them. But this is exactly the behavior that you would expect from fractal traffic. The doubled capacity gave enough headroom for some of the peaks to get through, but not enough for all of them. On Ebone in Europe we used to have 40% as our threshold for upgrading core circuits.
I'm pushing an agenda in the open source world to add some concept of locality, with the purpose of moving traffic off ISP networks when I can. I think the user will be just as happy or happier, and folks pushing large optics will certainly be.
When you hear stories like the Icelandic ISP who discovered that P2P was 80% of their submarine bandwidth and promptly implemented P2P throttling, I think that the open source P2P will be driven to it by their user demand. --Michael Dillon
Current thread:
- RE: Extreme congestion (was Re: inter-domain link recovery), (continued)
- RE: Extreme congestion (was Re: inter-domain link recovery) Mikael Abrahamsson (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Deepak Jain (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Mikael Abrahamsson (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Fred Baker (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Mikael Abrahamsson (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Perry Lorier (Aug 18)
- Re: Extreme congestion (was Re: inter-domain link recovery) Mikael Abrahamsson (Aug 19)
- Re: Extreme congestion (was Re: inter-domain link recovery) Alexander Harrowell (Aug 21)
- Re: Extreme congestion (was Re: inter-domain link recovery) Mikael Abrahamsson (Aug 21)
- Re: Extreme congestion (was Re: inter-domain link recovery) Joe Provo (Aug 19)
- RE: Extreme congestion (was Re: inter-domain link recovery) michael.dillon (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Adrian Chadd (Aug 16)
- Re: Extreme congestion (was Re: inter-domain link recovery) Alexander Harrowell (Aug 17)
- Re: Extreme congestion (was Re: inter-domain link recovery) Stephen Wilcox (Aug 15)
- Re: Extreme congestion (was Re: inter-domain link recovery) ChiloƩ Temuco (Aug 15)
- RE: Extreme congestion (was Re: inter-domain link recovery) Rod Beck (Aug 15)
- Message not available
- Re: Extreme congestion (was Re: inter-domain link recovery) Stephen Wilcox (Aug 16)