Re: Lossy cogent p2p experiences?

[Nick Hilliard <nick () foobar org>]

Masataka Ohta wrote on 03/09/2023 14:32:
> See, for example, the famous paper of "Sizing Router Buffers".
>
> With thousands of TCP connections at the backbone recognized
> by the paper, buffers with thousands of packets won't cause
> packet reordering.
>
> > What you said reminds me of the old saying: in theory, there's no 
> > difference between theory and practice, but in practice there is.
>
> In theory, you can always fabricate unrealistic counter examples
> against theories by ignoring essential assumptions of the theories.
>
> In this case, "Without buffer bloat" is an essential assumption.

I can see how this conclusion could potentially be reached in specific 
styles of lab configs, but the real world is more complicated and the 
assumptions you've made don't hold there, especially the implicit ones. 
Buffer bloat will make this problem worse, but small buffers won't 
eliminate the problem.

That isn't to say that packet / cell spray arrangements can't work. 
There are some situations where they can work reasonably well, given 
specific constraints, e.g. limited distance transmission path and path 
congruence with far-side reassembly (!), but these are the exception. 
Usually this only happens inside network devices rather than between 
devices, but occasionally you see products on the market which support 
this between devices with varying degrees of success.

Generally in real world situations on the internet, packet reordering 
will happen if you use round robin, and this will impact performance for 
higher speed flows. There are several reasons for this, but mostly they 
boil down to a lack of control over the exact profile of the packets 
that the devices are expected to transmit, and no guarantee that the 
individual bearer channels have identical transmission characteristics. 
Then multiply that across the N load-balanced hops that each flow will 
take between source and destination.  It's true that per-hash load 
balancing is a nuisance, but it works better in practice on larger 
heterogeneous networks than RR.

Nick