nanog mailing list archives

Re: few big monolithic PEs vs many small PEs


From: Mark Tinka <mark.tinka () seacom mu>
Date: Thu, 27 Jun 2019 17:26:03 +0200



On 27/Jun/19 10:58, James Bensley wrote:

Hi Adam,

Over the years I have been bitten multiple times by having fewer big
routers with either far too many services/customers connected to them
or too much traffic going through them. These days I always go for
more smaller/more routers than fewer/larger routers.

One experience I have made is that when there is an outage on a large
PE, even when it still has spare capacity, is that the business impact
can be too much to handle (the support desk is overwhelmed, customers
become irate if you can't quickly tell them what all the impacted
services are, when service will be restored, the NMS has so many
alarms it’s not clear what the problem is or where it's coming from
etc.).

I’ve seen networks place change freeze on devices, with the exception
of changes that migrate customers or services off of the PE, because
any outage would create too great an impact to the business, or risk
the customers terminating their contract. I’ve also seen changes
freeze be placed upon large PEs because the complexity was too great,
trying to work out the impact of a change on one of the original PEs
from when the network was first built, which is somehow linked to
virtually every service on the network in some obscure and
unforeseeable way.

I would tend to agree when the edge routers are massive, e.g., boxes
like the Cisco ASR9922 or the Juniper MX2020 are simply too large, and
present a real risk re: that level of customer aggregation (even for
low-revenue services such as Broadband). I don't think I'd ever justify
buying these towers to aggregate customers, mainly due to the risk.

For us, even the MX960 is too big, which is why we focus on the MX480
(ASR9906 being the equivalent). It's a happy medium between the small
and large end of the spectrum.

And as I mentioned before, we just look at a totally different box for
100Gbps customers.

Mark.


Current thread: