Re: The Making of a Router

[Blake Dunlap <ikiris () gmail com>]

Pretty much what everyone else said. I'm a huge linux person, almost
everything I use is linux, run full Myth set up etc, but I wouldn't use it
for a high PPS situation like this. It's just asking for suffering later,
at the worst possible times.

-Blake


On Sat, Dec 28, 2013 at 9:45 AM, Shawn Wilson <ag4ve.us () gmail com> wrote:

> Chris Adams <cma () cmadams net> wrote:
> > Once upon a time, Shawn Wilson <ag4ve.us () gmail com> said:
> > > I was hoping someone could give technical insight into why this is
> > good or not and not just "buy a box branded as a router because I said
> > so or your business will fail". I'm all for hearing about the business
> > theory of running an ISP (not my background or day job) but didn't
> > think that's what the OP was asking about (and it didn't seem they were
> > taking business suggestions very well anyway).
> >
> > There's been some technical insight here I would say.  I'm a big Linux,
> > Open Source, and Free Software advocate, and I'll use Linux-based
> > systems for routing/firewalling small stuff, but for high speed/PPS,
> > get
> > a router with a hardware forwarding system (I like Juniper myself).
> >
> > You can build a decently-fast Linux (or *BSD) system, but you'll need
> > to
> > spend a good bit of time carefully choosing motherboards, cards, etc.
> > to
> > maximize packet handling, possibly buying multiple of each to find the
> > best working combination.  Make sure you buy a full set of spares once
> > you find a working combination (because in the PC industry, six months
> > is a lifetime).  Then you have to build your OS install, tweaking the
> > setup, network stack, etc.
> >
> > After that, you have to stay on top of updates and such (so plan for
> > more reboots); while on a hardware-forwarding router you can mostly
> > partition off the control plane, on a Linux/*BSD system, the base OS is
> > the forwarding plane.  Also, if something breaks, falls over under an
> > attack, etc., you're generally going to be on your own to figure it
> > out.
> > Maybe you can Google the answer (and hope it isn't "that'll be fixed in
> > kernel 3.<today's version+2>.  Not saying that doesn't happen with
> > router vendors (quoting RFCs at router engineers is "fun"), but it is
> > IMHO less often.
> >
> > The question becomes: what is your time worth?  You could spend
> > hundreds
> > of hours going from the start to your satisfactory in-service router,
> > and have a potentially higher upkeep cost.  Can you hire somebody with
> > all the same Linux/*BSD knowlege as yourself, so you are not on-call
> > for
> > your home-built router around the clock?
> >
> > I've used Linux on all my computers for almost 20 years, I develop on
> > Linux, and contribute to a Linux distribution.  However, when I want to
> > record TV to watch later, I plug in a TiVo, not build a MythTV box.
> > There is a significant value in "just plug it in and it works", and if
> > you don't figure your time investment (both up-front and on-going) into
> > the cost, you are greatly fooling yourself.
>
> I agree with all of this to some degree. IDK whether cost of ownership on
> a hardware router or a desktop is more or less - I jus haven't done the
> research. We use them at work and at home I have Cisco and Linksys gear
> (plus Linux doing some things the router could like DHCP) - go figure.
>
> I agree that some network cards and boards work better than others (and am
> partial to the Intel Pro cards - though I'm unsure if they're still the
> best). I would also hesitate to route that much traffic with a PC. Though,
> I have no technical reason for this bias.
>
> If you have hardware in production, you really should have a spare -
> whether we're talking servers, HDDs, batteries, or routers. Ie, that
> comment is not unique to servers. I also don't think warranty has any
> bearing on this - I've seen servers stay down for over a day because (both
> HP and Dell for their respective hardware) screwed up and the company
> didn't budget for a spare board and I've seen a third of a network be taken
> out because multiple switch ports just died. How much would a spare switch
> have cost compared to 50 people not online?
>
> At any rate, I'm interested in this because I've worked in both
> environments and haven't seen a large difference between the two approaches
> (never worked at an ISP or high bandwidth web environment though). I do
> like the PC router approach because it allows more versatility wrt dumping
> packets (no need to dig out that 10mbit dumb hub and throttle the whole
> network), I can run snort or do simple packet inspection with iptables
> (some routers can do this but most can't or require a license). So I'm
> sorta leaning to the PC router as being better - maybe not cheaper but
> better.