Re: gigabit router (was Re: Getting a "portable" /19 or /20)

[Matt Zimmerman <mdz () csh rit edu>]

On Thu, Apr 12, 2001 at 10:16:00AM -0400, Craig Partridge wrote:

> Having implemented distributed forwarding once, and consulted on it a couple
> of times, the answer is that it adds complexity but not where you'd think.
> [...]
> Now I just said "all you need" -- that's a lie :-).  The major complexity is
> actually getting the forwarding tables out to the inbound interfaces.  There
> are at least two issues: (a) bandwidth [a small forwarding table, times 32 or
> 64 ports rapidly becomes big, especially for the control data path which is
> typically short on bandwidth] and (b) finding a way to update the forwarding
> table in a way that is not too disruptive to forwarding [incremental updates
> are often mindbreakingly hard to do, esp. if the update changes the table
> size, or invalidates some entries -- you can have the forwarding table doing
> housecleaning instead of forwarding].  We used a bank swap -- we had an
> active forwarding table and an inactive one.  Updates were sent to the
> inactive bank and when you had a fully updated bank, you send a message to
> the forwarding processor saying "swap banks."  And, of course, the software
> to track which forwarding engine had which bank live and the current state of
> each bank's updates got a little exciting.

Yes, I had thought about this kind of consistency as well, within a single
forwarding table.  You have to provide a transactional system to update a
complete table (or a self-contained subset) in an atomic operation.

In practice, many of the forwarding tables for different interfaces will
contain identical entries, correct?  Most traffic for a given destination will
leave through one interface, with the same next-hop.  Is it possible (or
useful) to attempt to send forwarding table entries (or chunks) across the bus
to multiple interfaces at once, to approach the bandwidth issue?

Thanks for sharing your experience with the list.

-- 
 - mdz