Re: The Making of a Router

[Shawn Wilson <ag4ve.us () gmail com>]

Chris Adams <cma () cmadams net> wrote:
> Once upon a time, Shawn Wilson <ag4ve.us () gmail com> said:
> > I was hoping someone could give technical insight into why this is
> good or not and not just "buy a box branded as a router because I said
> so or your business will fail". I'm all for hearing about the business
> theory of running an ISP (not my background or day job) but didn't
> think that's what the OP was asking about (and it didn't seem they were
> taking business suggestions very well anyway).
>
> There's been some technical insight here I would say.  I'm a big Linux,
> Open Source, and Free Software advocate, and I'll use Linux-based
> systems for routing/firewalling small stuff, but for high speed/PPS,
> get
> a router with a hardware forwarding system (I like Juniper myself).
>
> You can build a decently-fast Linux (or *BSD) system, but you'll need
> to
> spend a good bit of time carefully choosing motherboards, cards, etc.
> to
> maximize packet handling, possibly buying multiple of each to find the
> best working combination.  Make sure you buy a full set of spares once
> you find a working combination (because in the PC industry, six months
> is a lifetime).  Then you have to build your OS install, tweaking the
> setup, network stack, etc.
>
> After that, you have to stay on top of updates and such (so plan for
> more reboots); while on a hardware-forwarding router you can mostly
> partition off the control plane, on a Linux/*BSD system, the base OS is
> the forwarding plane.  Also, if something breaks, falls over under an
> attack, etc., you're generally going to be on your own to figure it
> out.
> Maybe you can Google the answer (and hope it isn't "that'll be fixed in
> kernel 3.<today's version+2>.  Not saying that doesn't happen with
> router vendors (quoting RFCs at router engineers is "fun"), but it is
> IMHO less often.
>
> The question becomes: what is your time worth?  You could spend
> hundreds
> of hours going from the start to your satisfactory in-service router,
> and have a potentially higher upkeep cost.  Can you hire somebody with
> all the same Linux/*BSD knowlege as yourself, so you are not on-call
> for
> your home-built router around the clock?
>
> I've used Linux on all my computers for almost 20 years, I develop on
> Linux, and contribute to a Linux distribution.  However, when I want to
> record TV to watch later, I plug in a TiVo, not build a MythTV box.
> There is a significant value in "just plug it in and it works", and if
> you don't figure your time investment (both up-front and on-going) into
> the cost, you are greatly fooling yourself.

I agree with all of this to some degree. IDK whether cost of ownership on a hardware router or a desktop is more or 
less - I jus haven't done the research. We use them at work and at home I have Cisco and Linksys gear (plus Linux doing 
some things the router could like DHCP) - go figure.

I agree that some network cards and boards work better than others (and am partial to the Intel Pro cards - though I'm 
unsure if they're still the best). I would also hesitate to route that much traffic with a PC. Though, I have no 
technical reason for this bias. 

If you have hardware in production, you really should have a spare - whether we're talking servers, HDDs, batteries, or 
routers. Ie, that comment is not unique to servers. I also don't think warranty has any bearing on this - I've seen 
servers stay down for over a day because (both HP and Dell for their respective hardware) screwed up and the company 
didn't budget for a spare board and I've seen a third of a network be taken out because multiple switch ports just 
died. How much would a spare switch have cost compared to 50 people not online?

At any rate, I'm interested in this because I've worked in both environments and haven't seen a large difference 
between the two approaches (never worked at an ISP or high bandwidth web environment though). I do like the PC router 
approach because it allows more versatility wrt dumping packets (no need to dig out that 10mbit dumb hub and throttle 
the whole network), I can run snort or do simple packet inspection with iptables (some routers can do this but most 
can't or require a license). So I'm sorta leaning to the PC router as being better - maybe not cheaper but better.