Re: Anycast 101

[Paul Vixie <vixie () vix com>]

vixie () vix com (Paul Vixie) (hey, that's me!) wrote:

> > > as i said the other day, "all power tools can kill."  if you turn
> > > on PPLB and it hurts, then turn it off until you can read the
> > > manual or take a class or talk to an expert.  PPLB is a link
> > > bundling technology.  if you turn it on in non-parallel-path
> > > situation, it will hurt you, so, "don't do that."

Iljitsch replied as follows:

> > Yes, per packet load balancing will cause reordering, and if that's
> > an issue you shouldn't use it.  But if with pplb packets end up at
> > two different hosts, that's not the fault of the people who invented
> > per packet load balancing or the people who turned it on, but the
> > fault of the people giving the same address to two different hosts.

i then bypassed Iljitsch and went to the gallery:

> i would be interested in hearing from anybody else who thinks that
> turning on pplb in a eyeball-centric isp that has multiple upstream
> paths is a reasonable thing to do, even if there were no anycast
> services deployed anywhere in the world.

so far, no takers.  i've heard from rfc-writers who say pplb was never
meant to be used the way Iljitsch is describing it, and i've heard from
equipment vendors who say their customers don't do that and that if some
customer did that and asked for support the response would be "don't do
that!", and i've heard from network operators who say they would never do
that, and i've heard from customers of network operators who did that with
notable bad effects.

but so far nobody has said "yes, what Iljitsch is describing should work."

let me summarize.  Iljitsch says that pplb is incompatible with anycast,
since a pplb-using access router at the inner edge of an ISP could hear
two different IGP routes to some destination, which ended up taking 
different exits from the ISP and thus different BGP paths.  whereas pplb
would normally only operate on equal-cost paths, the BGP->IGP path would
hide the variance in BGP paths and make these "paths" eligible for pplb.

i've said that pplb is only useful for turning two OC3's into an "OC6" (or
similar circuit bundling where a pair of routers has multiple connections
to eachother) and that even in this case, packet reordering is likely to
occur, which will make tcp-flow performance suffer across this "link".  i
have also said that turning pplb on across non-parallel links, such as to
multiple providers or through multiple tunnels or whatever, would pretty
much guaranty that a word rhyming with "massive suckage" would occur.  and
i've made these claims independent of anycast -- that is, life will be bad
if you use pplb outside its intended purpose, even if nobody anywhere was
using anycast.

loath though i am to treat a "preponderance of assertion" as equivilent
to "proof", i see no alternative on this issue.  noone is defending the use
case Iljitsch is proposing.  noone is even saying "i tried that and it was
OK".  lots of people are saying various things like "don't do that!" and
"are you crazy?"

it's important to point out a third time that it's indeed possible that
Iljitsch's proposed use case for pplb would interact badly with anycast,
and that i'm not arguing against that assertion.  i'm saying that the pplb
configuration proposed by Iljitsch would have really bad consequences even
if noone, anywhere on the internet, was using anycast.

and so we return to yesterday's statement:

> at the moment i am completely certain that turning on pplb would be an
> irrational act, and would have a significant performance-dooming
> effect on a client population behind it, and that the times when pplb
> would actually be useful and helpful are very limited, and that
> anycast doesn't even enter into the reasons why doing as Iljitsch
> paints would be a bad idea.

and i'll repeat, again:

> but my mind is open, if anyone can speak from experience on the matter.

and, "good luck storming the castle, boys."
-- 
Paul Vixie