nanog mailing list archives
Re: Facebook post-mortems...
From: Bjørn Mork <bjorn () mork no>
Date: Wed, 06 Oct 2021 08:56:33 +0200
Masataka Ohta <mohta () necom830 hpcl titech ac jp> writes:
As long as name servers with expired zone data won't serve request from outside of facebook, whether BGP routes to the name servers are announced or not is unimportant.
I am not convinced this is true. You'd normally serve some semi-static content, especially wrt stuff you need yourself to manage your network. Removing all DNS servers at the same time is never a good idea, even in the situation where you believe they are all failing. The problem is of course that you can't let the servers take the decision to withdraw from anycast if you want to prevent this catastrophe. The servers have no knowledge of the rest of the network. They only know that they've lost contact with it. So they all make the same stupid decision. But if the servers can't withdraw, then they will serve stale content if the data center loses backbone access. And with a large enough network then that is probably something which happens on a regular basis. This is a very hard problem to solve. Thanks a lot to facebook for making the detailed explanation available to the public. I'm crossing my fingers hoping they follow up with details about the solutions they come up with. The problem affects any critical anycast DNS service. And it doesn't have to be as big as facebook to be locally critical to an enterprise, ISP or whatever. Bjørn
Current thread:
- Re: Better description of what happened, (continued)
- Re: Better description of what happened Curtis Maurand (Oct 06)
- Re: Better description of what happened PJ Capelli via NANOG (Oct 06)
- Re: Better description of what happened Andy Brezinsky (Oct 05)
- Re: Better description of what happened Michael Thomas (Oct 05)
- Re: Better description of what happened Hugo Slabbert (Oct 05)
- Re: Better description of what happened Tom Beecher (Oct 06)
- Re: Better description of what happened Bjørn Mork (Oct 06)
- Re: Better description of what happened Tom Beecher (Oct 06)
- Re: Better description of what happened Hugo Slabbert (Oct 06)
- Re: Facebook post-mortems... Masataka Ohta (Oct 05)
- Re: Facebook post-mortems... Bjørn Mork (Oct 05)
- Re: Facebook post-mortems... Masataka Ohta (Oct 06)
- Re: Facebook post-mortems... Bjørn Mork (Oct 06)
- DNS pulling BGP routes? Michael Thomas (Oct 06)
- Re: DNS pulling BGP routes? J. Hellenthal via NANOG (Oct 06)
- Re: DNS pulling BGP routes? Jared Mauch (Oct 06)
- Re: DNS pulling BGP routes? Blake Dunlap (Oct 06)
- Re: DNS pulling BGP routes? Masataka Ohta (Oct 06)
- Re: DNS pulling BGP routes? William Herrin (Oct 07)
- Re: DNS pulling BGP routes? Masataka Ohta (Oct 07)
- Re: DNS pulling BGP routes? Tom Beecher (Oct 07)