nanog mailing list archives

Re: FB?


From: cosmo <clinton.mielke () gmail com>
Date: Thu, 14 Mar 2019 14:13:36 -0700

Looks like Google recently posted their post-mortem of their outage on the
12th
https://status.cloud.google.com/incident/storage/19002

On Thu, Mar 14, 2019 at 1:21 PM Suresh Ramasubramanian <ops.lists () gmail com>
wrote:

That's a 2010 outage that someone dug out and was doing the rounds as a
new one

--srs

------------------------------
*From:* NANOG <nanog-bounces () nanog org> on behalf of cosmo <
clinton.mielke () gmail com>
*Sent:* Thursday, March 14, 2019 9:50 PM
*To:* Bryan Holloway
*Cc:* nanog () nanog org
*Subject:* Re: FB?

Facebook pushed an update to their code that manages cookies, that had a
rather severe bug in it that resulted in a large flood of requests to their
database servers. To deal with this load, they had to prevent all writes
and then slowly allow people back on.

I saw the writeup for it last night but cannot seem to find it now! Grrr.
Did I dream it?

On Thu, Mar 14, 2019 at 8:42 AM Bryan Holloway <bryan () shout net> wrote:


On 3/14/19 9:06 AM, Tom Beecher wrote:
As much as I wanted to crack jokes because I cannot stand Facebook (the
product), much love to all you FB engineers that went through (and are
probably still going through) much hell.


+1 on both counts.

We've all been there; no bueno.



Current thread: