Re: Gigabit Linux Routers

["Scott Francis" <darkuncle () gmail com>]

the recent facebook engineering post on scaling memcached to 200-300K
UDP requests/sec/node may be germaine here (in particular, patches to
make irq handling more intelligent become very useful at the traffic
levels being discussed).
http://www.facebook.com/note.php?note_id=39391378919&id=9445547199&index=0

/sf

On Wed, Dec 17, 2008 at 8:30 AM, Jim Shankland <nanog () shankland org> wrote:
> Chris wrote:
> > Hi All,
> > Sorry if this is a repeat topic. I've done a fair bit of trawling but
> > can't
> > find anything concrete to base decisions on.
> >
> > I'm hoping someone can offer some advice on suitable hardware and kernel
> > tweaks for using Linux as a router running bgpd via Quagga. We do this at
> > the moment and our box manages under the 100Mbps level very effectively.
> > Over the next year however we expect to push about 250Mbps outbound
> > traffic
> > with very little inbound (50Mbps simultaneously) and I'm seeing differing
> > suggestions of what to do in order to move up to the 1Gbps level.
>
> As somebody else said, it's more pps than bits you need to worry about.
> The Intel NICs can do a full gigabit without any difficulty, if packet
> size is large enough.  But they buckle somewhere around 300Kpps.  300K
> 100-byte packets is only 240 Mb/s.  On the other hand, you mentioned
> your traffic is mostly outbound, which makes me think you might be a
> content provider.  In that case, you'll know what your average packet
> size is -- and it should be a lot bigger than 100 bytes.  For that type
> of traffic, using a Linux router up to, say, 1.5-2 Gb/s is pretty trivial.
> You can do more than that, too, but have to start getting a lot more careful
> about hardware selection, tuning, etc.
>
> The other issue is the number of concurrent flows.  The actual route
> table size is unimportant -- it's the size of the route cache that
> matters.  Unfortunately, I have no figures here.  But I did once
> convert a router from limited routes (quagga, 10K routes) to full routes
> (I think about 200K routes at the time), with absolutely no measurable
> impact.  There were only a few thousand concurrent flows, and that
> number did not change -- and that's the one that might have made a
> difference.
>
> I hope this is helpful.
>
> Jim



-- 
darkuncle@{gmail.com,darkuncle.net} || 0x5537F527
  http://darkuncle.net/pubkey.asc for public key