nanog mailing list archives
Re: Facebook post-mortems...
From: Joe Maimon <jmaimon () jmaimon com>
Date: Tue, 5 Oct 2021 12:44:05 -0400
Mark Tinka wrote:
You could put up a temp page or two. Like, the internet is not down, we are just having a bad day. Bear with us for a bit. Go outside and enjoy nature for the next few hours.So I'm not worried about DNS stability when split across multiple physical entities.I'm talking about the actual services being hosted on a single network that goes bye-bye like what we saw yesterday.All the DNS resolution means diddly, even if it tells us that DNS is not the issue.Mark.
But more importantly, internal infrastructure domains, containing router names, bootstraps, tools, utilities, physical access control, config repositories, network documentations, oob-network names (who remembers those?) , oob-email, oob communications (messenger, conferences, voip), etc..
Doesnt even have to be globally registered. External DNS server in the resolver list of all tech laptops slaving the zone.
Rapid response requires certain amenities, or as we can see, your talking about hours just getting started.
Also, the oob-network needs to be used regularly or it will be essentially unusable when actually needed, due to bit rot (accumulation of unnoticed and unresolved issues) and lack of mind muscle memory.
It should be standard practice to deploy all new equipment from the oob-network servicing it. Install things how you want to be able to repair them.
Joe
Current thread:
- Re: Facebook post-mortems..., (continued)
- Re: Facebook post-mortems... Mark Tinka (Oct 05)
- Re: Facebook post-mortems... - Update! Mark Tinka (Oct 05)
- Re: Facebook post-mortems... - Update! Warren Kumari (Oct 05)
- BGP communities, was: Re: Facebook post-mortems... - Update! Jay Hennigan (Oct 05)
- Re: BGP communities, was: Re: Facebook post-mortems... - Update! Ross Tajvar (Oct 07)
- Re: Facebook post-mortems... - Update! Randy Bush (Oct 05)
- Re: Facebook post-mortems... Joe Greco (Oct 05)
- Re: Facebook post-mortems... Mark Tinka (Oct 05)
- RE: Facebook post-mortems... Jean St-Laurent via NANOG (Oct 05)
- Re: Facebook post-mortems... Mark Tinka (Oct 05)
- Re: Facebook post-mortems... Joe Maimon (Oct 05)
- Re: Facebook post-mortems... Matthew Kaufman (Oct 05)
- Re: Facebook post-mortems... Mark Tinka (Oct 05)
- RE: Facebook post-mortems... Jean St-Laurent via NANOG (Oct 05)
- Re: Facebook post-mortems... Ryan Brooks (Oct 05)
- Re: Facebook post-mortems... Tom Beecher (Oct 05)
- Re: Facebook post-mortems... Lou D (Oct 05)
- Re: Facebook post-mortems... Niels Bakker (Oct 05)
- Re: Facebook post-mortems... Tom Beecher (Oct 05)
- RE: Facebook post-mortems... Jean St-Laurent via NANOG (Oct 05)
- Re: Facebook post-mortems... Ryan Landry (Oct 05)