Re: Bottlenecks and link upgrades

["Radu-Adrian Feurdean" <nanog () radu-adrian feurdean net>]

On Sat, Aug 15, 2020, at 11:35, Baldur Norddahl wrote:
> No plan survives contact with the enemy. Your careful made growth 
> projection was fine until the brass made a deal with some major 
> customer, which caused a traffic spike. 

Capacity planning also includes keeping an eye on what is being sold and what is being prepared.
Having the traffic more than double within a 48h timespan (until day X peak at N Gbps, after days X+2, peaks at 2.5*N 
Gbps) -> done with success when the correct information ("partner X will change delivery system") arrived 4 months in 
advance.

Having multiple 200 Mbps and 500 Mbps connections over an already-used 1 Gbps port and pretending that "everything's 
gonna be allright" , in that case you should confront your enemy.

> Or any infinite other events that could and eventually will happen to you.

Among which you try to protect yourself against the most realistic ones.
 
> One hard thing, that almost everyone will get wrong at some point, is 
> simulating load in the event multiple outages takes some links out, 
> causing excessive traffic to reroute unto links that previously seemed 
> fine.

You should scale the network to absorb a certain degree of "surprise"/damage, and clearly explain that beyond that 
certain level, service will be degraded (or even absent) and there is nothing that can and nothing that will be done 
immediately.

Every network fails at a certain moment in time. You just need to make sure you know how to make it working again, 
within a reasonable time frame. Or have a good run-away plan (sometimes this is the best solution).