Re: massive facebook outage presently

[Mark Tinka <mark@tinka.africa>]

On 10/4/21 20:48, Luke Guillory wrote:

> I believe the original change was 'automatic' (as in configuration 
> done via a web interface). However, now that connection to the outside 
> world is down, remote access to those tools don't exist anymore, so 
> the emergency procedure is to gain physical access to the peering 
> routers and do all the configuration locally.

Q: What is automation?
A: Breaking the network at scale.

I suppose we shall know more in the coming days, but to me, it smells 
like a wide-scale automatic update roll-out that didn't go to plan. 
Isn't the first time a major content provider has suffered this; likely 
won't be the last.

In the end, the best thing for all of us is that it is a teaching 
moment, and we move the needle forward.

Mark.