nanog mailing list archives
Re: 10GE router resource
From: Andrew C Burnette <acb () acb net>
Date: Thu, 27 Mar 2008 01:32:26 -0400
William Herrin wrote:
On Wed, Mar 26, 2008 at 4:26 PM, Sargun Dhillon <sdhillon () decarta com> wrote:from a viewpoint of hardware, x86 is a fairly decent platform. I can stuff 40 (4x10GigE multiplex with a switch) 1 GigE ports in it. Though, the way that Linux works, it cannot handle high packet rates.Correction: The way DRAM works, it cannot handle high packet rates. Also note that the PCI-X bus tops out in the 7 to 8 gbps range and it's half-duplex.
Indeed. PCI-X is already an EOL'ed interface, if only cheap PCI-X cards were available. Once you add extensive ACL's, there's loads more [central] processing to be done than just packet routing (100k choices versus 2 to 4 interfaces). System throughput gets slammed rather quickly. Linux IPtables grumbles painfully at 100k line ACLs :) Not to mention the options of what to do with a packet are very limited.
The AMD chips with extra L1 cache perform better on *bsd platforms as the forwarding code is tight and likes to stay close to the CPU, and context switching kills packet processing performance (thus the small but notable increase in the multicore performance). The GP registers on the AMD platform are also easy to deal with (and in 64 bit mode, you get double the number for free) essentially working an end around a broken stack architecture from a few decades ago....anyone recall the simplicity of assembly language of the 6800 or the 6502? :-)
getting the latency down low enough for HPC clusters is a major hassle, as the x86 PC design just doesn't have the bandwidth.
Of course, Intel makes some slick NPU's for custom work (e.g. cloudshield.com). If you like starting at bit 0. (isn't that like slot zero or port zero, it technically doesn't exist since zero is only a placeholder in larger numbers if you mean anything greater than none? I could swear back in the days of a SLC96, ports were 1-96, not 0-95 :-) )
http://developer.intel.com/design/network/products/npfamily/index.htm?iid=ncdcnav2+proc_netproctoo bad they [Intel] don't make a hypertransport capable version, or you'd have one helluva multicore multiNPU system with no glue logic required.
Fun to play around though. regards, andy
High-rate routers try to keep the packets in an SRAM queue and instead of looking up destinations in a DRAM-based radix tree, they use a special memory device called a TCAM. http://www.pagiamtzis.com/cam/camintro.html Regards. Bill Herrin
Current thread:
- Re: 10GE router resource, (continued)
- Re: 10GE router resource Mark Tinka (Mar 26)
- Re: 10GE router resource Paul Vixie (Mar 26)
- Re: 10GE router resource Peter Wohlers (Mar 26)
- Re: 10GE router resource Robert Bays (Mar 26)
- Re: 10GE router resource Sargun Dhillon (Mar 26)
- Re: 10GE router resource William Herrin (Mar 26)
- Re: 10GE router resource Sargun Dhillon (Mar 26)
- Re: 10GE router resource William Herrin (Mar 26)
- RE: 10GE router resource michael.dillon (Mar 26)
- RE: 10GE router resource Buhrmaster, Gary (Mar 26)
- Re: 10GE router resource Andrew C Burnette (Mar 26)
- Re: 10GE router resource Adrian Chadd (Mar 26)
- Re: 10GE router resource Christopher Morrow (Mar 25)
- Message not available
- Re: 10GE router resource Robert Boyle (Mar 25)
- Re: 10GE router resource Eddy Martinez (Mar 25)