Re: gigabit router (was Re: Getting a "portable" /19 or /20)

["Richard A. Steenbergen" <ras () e-gerbil net>]

On Tue, 10 Apr 2001, Craig Partridge wrote:

> In message <Pine.BSF.4.21.0104101753540.98098-100000 () overlord e-gerbil net>, "R
> ichard A. Steenbergen" writes:
>
> > Don't be absurd, I can walk into fry's and pick up a motherboard with
> > 64bit/66mhz PCI, some Netgear GA620's, and all the other components for a
> > 1GHz computer for under $1000.
>
> OK, so your bus has 4.2 Gb/s of bandwidth.  But, alas, you're in a PC
> so you have to copy each packet from the line card, into main memory,
> examine it, and push it back out to a line card.  So each packet consumes
> twice its size in bus bandwidth.  So 2 1 Gb/s line cards will consume
> 4 Gb/s backplane.  Assuming you can run the PCI at full rate (which in
> my experience is a big big if), you can connect two Ethernets.

Bad assumptions. Like I said, the Alteon Tigon 2 firmware is opensource,
and you don't HAVE to DMA the entire packet into main memory. You can
easily coalasce and preprocess packets on the card, transfer only packet
headers or smaller across the PCI bus, and then DMA between cards for the
rest of the payload. The limitation is the switch fabric, and poor
assumptions.

> Incidentally, this isn't the full story either.  You have to do a route
> lookup on each of those packets.  That's typically 5 to 10 memory
> accesses... 5 memory access times 1 Mpps per gigabit times 2 gigabits
> is 10 million lookups per second or 100 ns per lookup.  Allowing for
> time spent getting through the chip to the pins, you probably need 60 or
> 70ns DRAM, which is doable.  Except, oops!, that completely consumes
> your memory bandwidth... where are you going to find the cycles to
> get the packets in and out?

Try ~ 10ns ram which you can buy 256MB of for ~ $60-80
(www.pricewatch.com). At any rate, A raw 3 or 4 level mtrie FIB fully
populated with the real 100k+ routes on the internet consumes less then
900kb, and all the interesting parts fit in the L2 cache of a Celeron A
where you can do about 22,000 lookups per MHz. Hardly excessive memory
bandwidth. The packet ram on the gige cards is also very fast, and could
easily accomidate a dCEF approach.

> PS: Side note, this illustrates where router vendors earn their bucks.
> Find a way to move data over each bus only once (double your
> bandwidth!). Design your memory subsystems to keep packets and routing
> data separate (increase your memory bandwidth!).  Find a processor
> that doesn't waste cycles doing virtual memory (improve your memory
> access times!).  Oh yes, and then add hot board swap, a working BGP
> implementation (quick, where's Tony Li working these days:-)), a CLI,
> and a power subsystem for a CO, and you're in business.

The last remaining dominance is the switch fabric, and with people like
broadcom churning out 12Mpps switch fabrics on a single chip for a few
hundred dollars, you are a fool if you believe that money is going
anywhere other then to the 50,000 people supporting the 50 ancient
protocols, and straight to the bank. And don't even get me started on BGP
(Procket btw).

-- 
Richard A Steenbergen <ras () e-gerbil net>       http://www.e-gerbil.net/ras
PGP Key ID: 0x138EA177  (67 29 D7 BC E8 18 3E DA  B2 46 B3 D8 14 36 FE B6)