nanog mailing list archives

Re: massive facebook outage presently


From: Glenn Kelley <glenn@connectivity.engineer>
Date: Mon, 4 Oct 2021 19:18:22 -0500

This is why you should have Routers that are Firmware Defaulted to your own network.  ALWAYS

Be it Calix or even a Mikrotik which you have setup with Netboot - having these default to your own setup is REALLY a game changer.

Without it - you are rolling trucks or at minimum taking heavy call volumes.


Glenn Kelley
Chief cook and Bottle Washing Watcher @ Connectivity.Engineer

On 10/4/2021 3:56 PM, Matt Hoppes wrote:
Yes,
We've seen that.

On 10/4/21 4:33 PM, Eric Kuhnke wrote:
I am starting to see reports that in ISPs with very large numbers of residential users, customers are starting to press the factory-reset buttons on their home routers/modems/whatever, in an attempt to make Facebook work. This is resulting in much heavier than normal first tier support volumes. The longer it stays down the worse this is going to get.



On Mon, Oct 4, 2021 at 3:30 PM Jay Hennigan <jay () west net <mailto:jay () west net>> wrote:

    On 10/4/21 12:11, bzs () theworld com <mailto:bzs () theworld com> wrote:
     >
     > Although I believe it's generally true that if a company appears
     > prominently in the news it's liable to be attacked I assume because      > the miscreants sit around thinking "hmm, who shall we attack today oh
     > look at that shiny headline!" I'd hate to ascribe any altruistic
     > motivation w/o some evidence like even a credible twitter post (maybe
     > they posted that on FB? :-)

    I personally believe that the outage was caused by human error and not
    something malicious. Time will tell.

    However, if you missed the 60 Minutes piece, it was a former employee
    who spoke out with some rather powerful observations. I don't think
    that
    this type of worldwide outage was caused by an outside bad actor. It is
    certainly within the realm of possibility that it was an inside job.

    In other news:

https://twitter.com/disclosetv/status/1445100931947892736?s=20

    --     Jay Hennigan - jay () west net <mailto:jay () west net>
    Network Engineering - CCIE #7880
    503 897-8550 - WB6RDV



Current thread: