Re: SDN Internet Router (sir)

[Tom Beecher <beecher () beecher cc>]

> What I originally linked to (and another link or two contributed since
> then) seem to be people that already built that solution.


What's been shared isn't the same as what I described. faucet / sir are
doing the decision part and the signaling part together.

I am saying separate them. BGP doesn't know how to make the *decision* you
want, but it does know how to process *signals* that you send it.


On Fri, Jan 6, 2023 at 10:58 AM Mike Hammett <nanog () ics-il net> wrote:

> Right.
>
> Only I'm not the guy to build that solution.
>
> What I originally linked to (and another link or two contributed since
> then) seem to be people that already built that solution.
>
>
>
> -----
> Mike Hammett
> Intelligent Computing Solutions <http://www.ics-il.com/>
> <https://www.facebook.com/ICSIL>
> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> <https://www.linkedin.com/company/intelligent-computing-solutions>
> <https://twitter.com/ICSIL>
> Midwest Internet Exchange <http://www.midwest-ix.com/>
> <https://www.facebook.com/mdwestix>
> <https://www.linkedin.com/company/midwest-internet-exchange>
> <https://twitter.com/mdwestix>
> The Brothers WISP <http://www.thebrotherswisp.com/>
> <https://www.facebook.com/thebrotherswisp>
> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> ------------------------------
> *From: *"Tom Beecher" <beecher () beecher cc>
> *To: *"Mike Hammett" <nanog () ics-il net>
> *Cc: *"Mel Beckman" <mel () beckman org>, "NANOG" <nanog () nanog org>
> *Sent: *Friday, January 6, 2023 9:51:43 AM
> *Subject: *Re: SDN Internet Router (sir)
>
> Gotcha.
>
> Setup a Quagga/Bird box. Do your top talker analysis , use that box to
> inject the routes you deem important with communities.  On your routers ,
> create policy structure to only take a default plus those communities.
>
> Obviously lots of devils in the details of the implementation , but
> something like that is all you need to do.
>
> On Fri, Jan 6, 2023 at 10:29 Mike Hammett <nanog () ics-il net> wrote:
>
> > Maybe?
> >
> > I don't need any additional performance tests, though. Just watching
> > which prefixes are the top talkers and leaving the rest to default.
> >
> > I'm not looking at this to do what a BGP optimizier would do and find the
> > best tested path to the top talkers and then massage BGP to get it routed
> > that way. Determine the top talkers, then let BGP do its thing for those
> > top talkers.
> >
> > I don't want to manually say X traffic from Y POP manually goes here, but
> > I don't want to just leave it to default routing either. Something in the
> > middle.
> >
> >
> >
> > -----
> > Mike Hammett
> > Intelligent Computing Solutions <http://www.ics-il.com/>
> > <https://www.facebook.com/ICSIL>
> > <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> > <https://www.linkedin.com/company/intelligent-computing-solutions>
> > <https://twitter.com/ICSIL>
> > Midwest Internet Exchange <http://www.midwest-ix.com/>
> > <https://www.facebook.com/mdwestix>
> > <https://www.linkedin.com/company/midwest-internet-exchange>
> > <https://twitter.com/mdwestix>
> > The Brothers WISP <http://www.thebrotherswisp.com/>
> > <https://www.facebook.com/thebrotherswisp>
> > <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> > ------------------------------
> > *From: *"Tom Beecher" <beecher () beecher cc>
> > *To: *"Mike Hammett" <nanog () ics-il net>
> > *Cc: *"Mel Beckman" <mel () beckman org>, "NANOG" <nanog () nanog org>
> > *Sent: *Friday, January 6, 2023 9:16:19 AM
> >
> > *Subject: *Re: SDN Internet Router (sir)
> >
> > Thanks for this example.
> >
> > It sounds like you are describing egress peer engineering, but kinda in
> > reverse.
> >
> > In 'traditional' EPE, the routers have all the routes, and you are using
> > the external controller to perform the performance tests that matter to
> > you, and signal the network where to take the traffic based on those tests.
> >
> > It seems like you want to do the same thing , but instead of having the
> > controller signal the network where to carry bits, you want the controller
> > to signal the networks what routes are present, and direct the bits that
> > way.
> >
> > Do I have this right?
> >
> > On Thu, Jan 5, 2023 at 4:12 PM Mike Hammett <nanog () ics-il net> wrote:
> >
> > > I hesitated to get too specific in examples because someone is going to
> > > drag the conversation into the weeds.
> > >
> > > Let's take the the Dallas - New Orleans - Atlanta example where I have a
> > > connection from New Orleans to Dallas and a connection from New Orleans to
> > > Atlanta.
> > >
> > > Let's say I peer with Netflix in both markets. Netflix chooses to serve
> > > me out of Atlanta, for whatever reason. Say my default route sends my
> > > traffic to Dallas. That's not where Netflix wanted it, so now I have to go
> > > from Dallas to Atlanta, whether that's my circuit or across the public
> > > Internet. Potentially, it's on MPLS and it rides back through the New
> > > Orleans router to get back to Atlanta. That's a long trip when I already
> > > had a better path, the less-than-full-fib router just didn't know about it.
> > > Given that Netflix is a sizable amount of traffic in an eyeball ISP, that's
> > > a lot of traffic to be going the wrong way. If the website for Viktor's
> > > Arctic Plunge in Siberia was hosted in Atlanta, I wouldn't give two craps
> > > that the traffic went the wrong way because A), I'll probably never go
> > > there and B) when someone does, it won't be meaningfully enough traffic to
> > > accommodate.
> > >
> > > Someone's going to tell me to put a full-table router in New Orleans.
> > > Maybe I should. Okay, so maybe I have a POP in Ashford, Alabama. It has
> > > transport to New Orleans and Atlanta. There aren't enough grains of sugar
> > > in Ashford, Alabama to justify a current-generation, full table router. Now
> > > I'm even closer to Atlanta, but default may point to New Orleans.
> > >
> > >
> > >
> > > -----
> > > Mike Hammett
> > > Intelligent Computing Solutions <http://www.ics-il.com/>
> > > <https://www.facebook.com/ICSIL>
> > > <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> > > <https://www.linkedin.com/company/intelligent-computing-solutions>
> > > <https://twitter.com/ICSIL>
> > > Midwest Internet Exchange <http://www.midwest-ix.com/>
> > > <https://www.facebook.com/mdwestix>
> > > <https://www.linkedin.com/company/midwest-internet-exchange>
> > > <https://twitter.com/mdwestix>
> > > The Brothers WISP <http://www.thebrotherswisp.com/>
> > > <https://www.facebook.com/thebrotherswisp>
> > > <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> > > ------------------------------
> > > *From: *"Mel Beckman" <mel () beckman org>
> > > *To: *"Mike Hammett" <nanog () ics-il net>
> > > *Cc: *"Joe Maimon" <jmaimon () jmaimon com>, "NANOG" <nanog () nanog org>
> > > *Sent: *Thursday, January 5, 2023 2:54:27 PM
> > > *Subject: *Re: SDN Internet Router (sir)
> > >
> > > Mike,
> > >
> > > I’m not sure I understand what you mean by “suboptimal“ routing. Even
> > > though the Internet uses AS path length for routing,  many of those path
> > > lengths are bogus, and don’t really represent any kind of path performance
> > > value. For example, a single AS might hide many hops in an MPLS network as
> > > a single hop, obscuring asymmetric routing and other uglies. Prepending
> > > also occurs when destinations are trying to enforce their own engineering
> > >  policies, which often conflict with yours or mine.
> > >
> > > So what do you mean by “suboptimal“? Are you thinking that the “best”
> > > path in BGP tables actually meant you were getting a performance benefit?
> > > Because that’s definitely not the case in today’s Internet. Were were you
> > > thinking that you would be going along less congested paths? That’s really
> > > at the mercy of the traffic engineering of backbone providers over which we
> > > have no control.
> > >
> > > I generally populate local router FIBs to merel choose an exit point for
> > > purposes of load balancing, and nothing more.
> > >
> > >  -mel
> > >
> > > On Jan 5, 2023, at 12:38 PM, Mike Hammett <nanog () ics-il net> wrote:
> > >
> > > 
> > > I guess I wasn't around for those days.
> > >
> > > As far as running out, again, assuming the tooling works correctly, I'd
> > > think to target fewer routes than you could hold. Maybe 1k routes is all
> > > one would need to get a significant percent of the traffic. A lot of room
> > > to mess up if you can hold 100k, 500k routes.
> > >
> > >
> > >
> > > -----
> > > Mike Hammett
> > > Intelligent Computing Solutions <http://www.ics-il.com/>
> > > <https://www.facebook.com/ICSIL>
> > > <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> > > <https://www.linkedin.com/company/intelligent-computing-solutions>
> > > <https://twitter.com/ICSIL>
> > > Midwest Internet Exchange <http://www.midwest-ix.com/>
> > > <https://www.facebook.com/mdwestix>
> > > <https://www.linkedin.com/company/midwest-internet-exchange>
> > > <https://twitter.com/mdwestix>
> > > The Brothers WISP <http://www.thebrotherswisp.com/>
> > > <https://www.facebook.com/thebrotherswisp>
> > > <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> > > ------------------------------
> > > *From: *"Joe Maimon" <jmaimon () jmaimon com>
> > > *To: *"Mike Hammett" <nanog () ics-il net>, "Christopher Morrow" <
> > > morrowc.lists () gmail com>
> > > *Cc: *"NANOG" <nanog () nanog org>
> > > *Sent: *Thursday, January 5, 2023 2:30:40 PM
> > > *Subject: *Re: SDN Internet Router (sir)
> > >
> > >
> > >
> > > Mike Hammett wrote:
> > > > I'm not concerned with which technology or buzzword gets the job done,
> > > > only that the job is done.
> > > >
> > > >
> > > >
> > > > Looking briefly at the couple of things out there, they're evaluating
> > > > the top X prefixes in terms of traffic reported by s-flow, where X is
> > > > the number I define, and those get pushed into the FIB. One
> > > > recalculates every hour, one does so more quickly. How much is
> > > > appropriate? I'm not sure. I can't imagine it would *NEED* to be done
> > > > all of that often, given the traffic/prefix density an eyeball network
> > > > will have. Default routes carry the rest. Default routes could be
> > > > handled outside of this process, such that if this process fails, you
> > > > just get some sub-optimal routing until repaired. Maybe it doesn't
> > > > filter properly and sends a bunch of routes. Then just have a prefix
> > > > limit set on the box. Maybe it sends the wrong prefixes. No harm, no
> > > > foul. If you're routing sub-optimally internally, when it does hit a
> > > > real router with a full FIB, it gets handled appropriately.
> > >
> > > Unless it loops.
> > >
> > > The rest sounds nice. But flow caching got a bad rap back in the early
> > > worm days. But thats because the situation was a little worse back then.
> > > Cache the wrong routes or run out of cache, router dies. So long as
> > > thats not the case automating optimization is an extremely valuable goal.
> > >
> > > > I would just be looking for solutions that influence what's in the FIB
> > > > and let the rest of the router work as the rest of the router would.
> > >
> > > The problem comes when the router wont work at all without the FIB
> > > routes, like in the olden days.
> > > > -----
> > > > Mike Hammett
> > > > Intelligent Computing Solutions <http://www.ics-il.com/>
> > > > <https://www.facebook.com/ICSIL><
> > > https://plus.google.com/+IntelligentComputingSolutionsDeKalb><
> > > https://www.linkedin.com/company/intelligent-computing-solutions><
> > > https://twitter.com/ICSIL>
> > > > Midwest Internet Exchange <http://www.midwest-ix.com/>
> > > > <https://www.facebook.com/mdwestix><
> > > https://www.linkedin.com/company/midwest-internet-exchange><
> > > https://twitter.com/mdwestix>
> > > > The Brothers WISP <http://www.thebrotherswisp.com/>
> > > > <https://www.facebook.com/thebrotherswisp><
> > > https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> > > >
> > > ------------------------------------------------------------------------
> > > > *From: *"Christopher Morrow" <morrowc.lists () gmail com>
> > > > *To: *"Mike Hammett" <nanog () ics-il net>
> > > > *Cc: *"Tom Beecher" <beecher () beecher cc>, "NANOG" <nanog () nanog org>
> > > > *Sent: *Thursday, January 5, 2023 12:27:08 PM
> > > > *Subject: *Re: SDN Internet Router (sir)
> > > >
> > > >
> > > >
> > > > On Thu, Jan 5, 2023 at 11:18 AM Mike Hammett <nanog () ics-il net
> > > > <mailto:nanog () ics-il net>> wrote:
> > > >
> > > >     Initially, my thought was to use community filtering to push just
> > > >     IXes, customers, and defaults throughout the network, but that's
> > > >     obviously still sub-optimal.
> > > >
> > > >     I'd be surprised if a last mile network had a ton of traffic going
> > > >     to any more than a few hundred prefixes.
> > > >
> > > >
> > > > I think in a low-fib box at the edge of your network your choices are:
> > > >   "the easy choice, get default, follow that"
> > > >
> > > >   "send some limited set of prefixes to the device, and default, so
> > > > you MAY choose better for the initial hop away"
> > > >
> > > > you certainly can do the second with communities, or route-filters
> > > > (prefix-list) on the senders, or....
> > > > you can choose what prefixes make the cut (get the community(ies))
> > > > based on traffic volumes or expected destination locality:
> > > >    "do not go east to go west!"
> > > >
> > > > these things will introduce toil and SOME suboptimal routing in some
> > > > instances... perhaps it's better than per flow choosing left/right
> > > > though and the support calls related to that choice.
> > > >
> > > > In your NOLA / DFW / ATL example it's totally possible that the
> > > > networks in question do something like:
> > > >   "low fib box in tier-2 city (NOLA), dfz capable/core devices in
> > > > tier-1 city (DFW/ATL), and send default from left/right to NOLA"
> > > >
> > > > Could they send more prefixes than default? sure... do they want to
> > > > deal with the toil that induces? (probably not says your example).
> > > >
> > > > SDN isn't really an answer to this, though.. I don't think. Unless you
> > > > envision that to lower the toil ?