Re: Whitebox Routers Beyond the Datasheet

[William Herrin <bill () herrin us>]

On Fri, Apr 12, 2024 at 6:03 AM Mike Hammett <nanog () ics-il net> wrote:
> What I've kind of come down to (based on little more than spec sheets)
> are the EdgeCore AGR400 and the UfiSpace S9600-30DX.

> That's wonderful and all, but does it take it from 64k routes
>  to 512k routes, or does it take it from 256k routes up to
>  the millions of routes?

Hi Mike,

You're combining two questions here. Break them apart. How many
distinct routes can its line-speed forwarding engine handle (the FIB)
and what processing/DRAM is available to handle the routing database
(RIB).

To take the EdgeCore AGR400, it has an 8-core Xeon and 32 gigs of ram.
It's going to handle a BGP RIB in the many millions of routes and a
BGP convergence time as fast as anything else you can buy.

For switching (FIB), it uses a Broadcom BCM88823. The data sheets on
the broadcom 88800 series chips are suspiciously light on information
about their internal capacity for forwarding decisions. Just a black
box in the diagram described as an "external lookup engine" and some
notes in the "packet processing" section that the ingress receive
packet processor "handles the main packet processing stage" and
"optionally" uses "expandable databases" from an external lookup
interface.

Large TCAMS are power hungry and the chip is physically small, so I'd
bet against it having an DFZ-size embedded TCAM. Smells like a switch
chip that typically has something in the single-digit thousands of
slots but that's strictly a guess. And with only 8 cores, any
software-switched "expandable databases" are going to be wimpy.

I'm not familiar with the specific hardware you mentioned, so none of
the above is certain. Just based on what I glean from the specs I
could find via google and my experience with white-box routing in
general. As a rule though, if the marketing materials don't call out a
component of the product as impressive, it probably isn't.

In general, white box routing is done with a framework like DPDK on
machines with large numbers of CPU cores. While they can handle
100gbps, they do it by running the cores in single-thread busywait
loops that eliminate the need for interrupts from the network devices.
This generates lots of heat and consumes lots of electricity.

Regards,
Bill Herrin


-- 
William Herrin
bill () herrin us
https://bill.herrin.us/