RE: gigabit router (was Re: Getting a "portable" /19 or /20)

["Kavi, Prabhu" <prabhu_kavi () tenornetworks com>]

Craig did a great job describing the problem and one valid approach to
solving it.  There are other valid approaches as well.  

Creating modern packet forwarding devices is based upon learning 
what did not work well in previous incarnations (such as how to 
synchronize these forwarding tables) and elmininating the problems 
(Too little bandwidth in the control plane?  Add bandwidth)
in the next version.  Many vendors learned about the synchronized 
forwarding table problem years ago, either with experience on their
own routers, or by getting talent from places (like BBN) that 
previously dealt with this.  And any recent system architecture created
by competent engineers should not have this problem.

Prabhu


> -----Original Message-----
> From: Craig Partridge [mailto:craig () aland bbn com]
> Sent: Thursday, April 12, 2001 10:16 AM
> To: Matt Zimmerman
> Cc: nanog () merit edu
> Subject: Re: gigabit router (was Re: Getting a "portable" /19 or /20) 
>
>
>
>
> In message <20010412100249.B15380 () alcor net>, Matt Zimmerman writes:
>
> > It _does_ show that distributed forwarding will be 
> significantly more complex
> > and more difficult to implement.  I am not arguing that 
> distributed forwarding
> > is a bad thing; on the contrary, it has a logical and 
> demonstrated performance
> > benefit.  However, it comes with a cost in overall system complexity.
>
> Having implemented distributed forwarding once, and consulted on it a
> couple of times, the answer is that it adds complexity but not where
> you'd think.
>
> You usually only have to generate one forwarding table (the outbound
> interface through which you send the packet doesn't change 
> depending on
> whether you make the decision centrally or at the inbound interfaces).
> You can try to reduce the size of the forwarding table by 
> excising stuff
> (e.g., info on VPNs that don't run through that interface) 
> but forwarding
> tables are pretty small so you don't have to do that if you 
> don't want to.
>
> Provided you don't do something stupid and have the outbound interface
> re-evaluate the inbound interface's routing decision, then 
> all you need
> is a way to copy the forwarding table to each interface (and if the
> interfaces receive the table at slightly staggered times that's OK,
> because they don't have to be perfectly consistent).
>
> Now I just said "all you need" -- that's a lie :-).  The 
> major complexity is
> actually getting the forwarding tables out to the inbound interfaces.
> There are at least two issues: (a) bandwidth [a small 
> forwarding table,
> times 32 or 64 ports rapidly becomes big, especially for the control
> data path which is typically short on bandwidth] and (b) 
> finding a way to
> update the forwarding table in a way that is not too 
> disruptive to forwarding
> [incremental updates are often mindbreakingly hard to do, esp. if the
> update changes the table size, or invalidates some entries -- you can
> have the forwarding table doing housecleaning instead of forwarding].
> We used a bank swap -- we had an active forwarding table and 
> an inactive
> one.  Updates were sent to the inactive bank and when you had a fully
> updated bank, you send a message to the forwarding processor saying
> "swap banks."  And, of course, the software to track which forwarding
> engine had which bank live and the current state of each 
> bank's updates
> got a little exciting.
>
> In short, generating the tables is easy, consistency isn't an issue,
> but getting the tables to the right place, in a timely 
> manner, and without
> getting confused and trashing a table, is a challenge.
>
> Thanks!
>
> Craig