nanog mailing list archives

Re: scaling linux-based router hardware recommendations


From: David bass <davidbass570 () gmail com>
Date: Mon, 26 Jan 2015 19:07:11 -0600

I'm also in the research stage of building our own router.  I'm interested in reading more if you can post links to 
some of this research and/or testing. 

David

Sent from my iPad

On Jan 26, 2015, at 6:45 PM, Phil Bedard <bedard.phil () gmail com> wrote:

Kind of unsurprisingly, the traditional network vendors are somewhat at 
the forefront of pushing what an x86 server can do as well.  Brocade 
(Vyatta), Juniper, and Alcatel-Lucent all have virtualized routers using 
Intel's DPDK pushing 5M+ PPS at this point.  They are all also tweaking 
what Intel is providing, and they are the ones with lots of software 
developers with a lot of hardware and network programming experience.  

ALU claims to be able to get 160Gbps full duplex through a 2RU server with 
16x10G interfaces and two 10-core latest-gen Xeon processors.  Of course 
that's probably at 9000 byte packet sizes, but at Imix type traffic it's 
probably still pushing 60-70Gbps.  They have a demo of lots of them in a 
single rack managed as a single router pushing Tbps.  

A commerical offering you are going to pay for that kind of performance 
and the control plane software.  Over time though you'll see the DPDK type 
enhancements make it into standard OS stacks.   Other options include 
servers with integrated network processors or NPs on a PCI card, there is 
a whole rash of those type of devices out there now and coming out.  

Phil 



On 1/26/15, 22:53, "micah anderson" <micah () riseup net> wrote:


Hi,

I know that specially programmed ASICs on dedicated hardware like Cisco,
Juniper, etc. are going to always outperform a general purpose server
running gnu/linux, *bsd... but I find the idea of trying to use
proprietary, NSA-backdoored devices difficult to accept, especially when
I don't have the budget for it.

I've noticed that even with a relatively modern system (supermicro with
a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server
adapters, and 16gig of ram, you still tend to get high percentage of
time working on softirqs on all the CPUs when pps reaches somewhere
around 60-70k, and the traffic approaching 600-900mbit/sec (during a
DDoS, such hardware cannot typically cope).

It seems like finding hardware more optimized for very high packet per
second counts would be a good thing to do. I just have no idea what is
out there that could meet these goals. I'm unsure if faster CPUs, or
more CPUs is really the problem, or networking cards, or just plain old
fashioned tuning.

Any ideas or suggestions would be welcome!
micah



Current thread: