Re: Anycast but for egress

[Baldur Norddahl <baldur.norddahl () gmail com>]

Here is what I think would happen if you were to try this setup. Let's
assume you deployed in eu-west-2 (London) and eu-central-1 (Frankfurt). You
would find that you could successfully connect to a number of networks but
also that some of them would work from the "wrong" site. Eg. you would have
clients in London that require you to use the Frankfurt instance to connect
to. You would also find a lot of networks that you could not connect to
from either site. And you would have some instability where some networks
at random switch between these states. For example you could have a client
that one day works from London, the next day it is Frankfurt and then
someday it won't be working at all.

As you add more sites, you also add more ways for the traffic to end up in
the wrong places. You could have something that works with just the two
sites above but then when you add eu-west-1 (Ireland) you could suddenly
not connect to them from any of the three sites.

There IS a possible solution which is to tunnel the traffic back to the
correct site. This still requires you to use unique IP addresses for each
site, but all addresses could be in the same subnet. You would have IP
a.b.c.1 to be London, a.b.c.2 Frankfurt, a.b.c.3 Ireland. Then if London
receives traffic to a.b.c.2 you would have a tunnel to send the traffic to
Frankfurt.

Regards,

Baldur


On Wed, Jul 28, 2021 at 11:07 AM Baldur Norddahl <baldur.norddahl () gmail com>
wrote:

> > > On Jul 27, 2021, at 17:20, Vimal <j.vimal () gmail com> wrote:
> > > Yes, this makes sense as the destination can be anywhere around the
> > world, and that routing is asymmetric as others mentioned.  However, if the
> > destination service is "close" (in the routing metric sense) to the
> > initiating host, anycast return IP ought to work well, right?  I understand
> > this is a very important caveat and impractical to implement correctly in
> > the real world.
>
> No, there is no such thing as "close". You could have a direct peering
> with some ISP and have them still deliver the responses on the other side
> of earth. You do not control the routing of other networks and can not be
> sure what they will do.
>
> For larger networks you may also have multiple peering points. Say you
> have a peering with them in city A and city B. How do you know which of
> their IP ranges are closer to A or B? You don't. And the same goes for
> them, they have no idea if you prefer A or B. Therefore you could select A
> and they may reply to B. They may even load balance between A and B if you
> are really unlucky.
>
> Routing is asymmetric. That means you have absolutely no idea where the
> replies end up going. Often it will not be what you think is "close".
>
> I do not run anycast, but I understand that the usual way of dealing with
> these issues is to do as little as possible with anycast before redirecting
> to a unicast address. For example you could have just your DNS on anycast
> and each site would reply with unique unicast addresses. Since DNS is just
> a single pair of UDP request/response, with the first packet originating
> from a unicast client, this works well.
>
> Regards,
>
> Baldur