Re: The questions stand

[Michael Dillon <michael () memra com>]

On Wed, 20 May 1998, Karl Denninger wrote:

> Part of the problem with "public exchanges" is that they get congested.  But
> the REASONS for that congestion are not, I believe, that well understood.

Some of the reasons are well understood. One is that when providers do not
upgrade the bandwidth of their pipes into the NAP then packets coming
towards them through the NAP get dropped. Thus they create congestion on
all these flows.

Another reason is that NAP architectures do not scale well past a certain
point. A NAP attempts to flatten the Interconnect hierarchy into a fully
meshed fabric but when the NAP traffic grows beyond the ability of a
specific full-mesh technology or device, then congestion occurs. For
instance, 10Mbps and 100Mbps Ethernet and 100 Mbps FDDI are all examples
of full-mesh technologies. But when the traffic exceeds their capacity,
congestion occurs. The Digital Gigaswitch is a device that can handle a
much higher traffic flow but when all the interface slots are full on a
single box congestion occurs. 

We can interconnect multiple devices and/or multiple technologies but now
there is no longer a full-mesh and much manual labor is involved in
adjusting things like trunk capacity. At this point we appear to be
attempting to collapse the entire Internet into an exchange point fabric
which is doomed to failure. In some ways, private interconnects appear to
be a superior technique as long as both providers keep them upgraded to
handle the traffic flows. The ideal exchange point for private
interconnects does not have a shared fabric, merely shared power and HVAC
in a building with lots of fiber ingress and no restrictions on cross
connecting.

But even private interconnects can run into a scaling issue because if we
attempt to interconnect every pair of providers through a private
interconnect, we are attempting to create a full mesh which is not
technically feasible. Thus we are led to a solution in which the largest
number of the smallest providers use fully-meshed fabrics to interconnect
and the larger providers manually build a full-mesh between themselves.

This reduces the problem to one of interconnecting the many fully-meshed
exchange point fabrics with the larger full-mesh fabric created by the
large providers and their private interconnects.

> Can we just build faster exchanges?  Sure.  Will it solve the problem?  Not
> if carriers don't provision fast enough circuits into them!

Karl is right. Faster exchanges may move various limits and bottlenecks
around but cannot solve the underlying problem. For instance, a faster
exchange raises the traffic level at which a mid-size provider MUST go to
private interconnects to handle traffic levels.

> If you're seeing poor performance between <X> and <Y> at an exchange, is 
> it due to the exchange fabric's poor performnace, or is one of <X> or <Y>
> under-provisoned into that fabric?  Have either of those carriers
> DELIBERATELY (or through negligence) failed to provide adequate 
> connectivity to the exchange?  

This is the same problem of bandwidth upgrades that I started my message
with. But it is also the heart of the issue of interconnecting the single
national mesh created by private interconnects with the many smaller
meshes created by the exchange points. There is a conflict of interest
when a major Internet backbone owns the exchange points because they can
attempt to deflect criticism of their connections to the exchange points
by pointing out that they are attempting to upgrade the bandwidth
capability of the exchange point. They fail to mention that growing the
exchange point is like the labors of Sisyphus and cannot succeed.

> Likewise, if the *EXCHANGE* operator was negligent (or just unable to keep
> up with demand) we could hold THEIR feet to the fire as operators.
>
> Sadly, I know of NO exchange currently in operation that subscribes to these
> operating rules and policies.

Not enough people really understand how the network mesh works to hold
anyone's feet to the fire. It doesn't matter whether you collapse portions
of the mesh into an exchange point or into a Gigaswitch; it's still the
same mesh. And you can even take the opposite tactic and expand an
exchange point mesh nationally but the mesh still has to handle the same
traffic levels. It is essentially a juggling game where you interconnect
various technologies that might be able to handle the traffic flows in
a given region of the mesh and hope that you don't drop too many balls.

--
Michael Dillon                 -               Internet & ISP Consulting
Memra Communications Inc.      -               E-mail: michael () memra com
http://www.memra.com           -  *check out the new name & new website*