nanog mailing list archives

Re: Never push the Big Red Button (New York City subway failure)


From: "Jay R. Ashworth" <jra () baylink com>
Date: Sun, 12 Sep 2021 03:23:41 +0000 (UTC)

----- Original Message -----
From: "Sean Donelan" <sean () donelan com>

NEW YORK CITY TRANSIT RAIL CONTROL CENTER POWER
OUTAGE ISSUE ON AUGUST 29, 2021
Key Findings
September 8, 2021

https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf

Key Findings
[...]

3. Based on the electrical equipment log readings and the manufacturer’s
official assessment, it was determined that the most likely cause of RCC
shutdown was the “Emergency Power Off” button being manually activated.

I don't even *do* datacenter for a living, and I know that when you hit the
Molly button, 

1) A Klaxon goes off in the Data Center -- one that sounds *different* from
the Halon Klaxon, in both cadence and tone (just for a couple bursts), and

2) Yellow rotating beacons turn on, and stay on while you're on Emergency Power.

Yes, real honest-to-ghod *rotating mechanical beacons*, none of this flashing LED
crap.

Clearly, it's important that the use of Emergency Power be annoyingly noticeable.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra () baylink com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates       http://www.bcp38.info          2000 Land Rover DII
St Petersburg FL USA      BCP38: Ask For It By Name!           +1 727 647 1274


Current thread: