nanog mailing list archives
Re: Never push the Big Red Button (New York City subway failure)
From: Fred Baker <fredbaker.ietf () gmail com>
Date: Wed, 15 Sep 2021 12:20:21 -0700
On Sep 10, 2021, at 1:33 PM, Warren Kumari <warren () kumari net> wrote: The utility let them know that they were going to be doing some maintenance work in the area. No impact expected, but out of an abundance of caution, they transfer over to generators. After the utility lets them know that the maintenance work is all finished, they want to switch back. If the generators are "emergency power", and you need to switch back to "utility power", obviously the way to do this must be the big red button, clearly marked as "EMERGENCY POWER OFF", no?!
One of the many stories that came out of 9/11 was a switching center in NY City that had a diesel generator as a power backup - which of course acted as primary when the city power is off. After a few days of operation, it needed to be refueled, so a truck was sent in carrying gasoline. The generator was refueled and restarted, and - oops - diesel != gasoline. So then they needed to bring in a new generator. Yup, it happens, and it happened.
Current thread:
- Never push the Big Red Button (New York City subway failure) Sean Donelan (Sep 10)
- RE: Never push the Big Red Button (New York City subway failure) Matthew Huff (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Christopher Morrow (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Chris Kane (Sep 10)
- RE: Never push the Big Red Button (New York City subway failure) Matthew Huff (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Baldur Norddahl (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Warren Kumari (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Tom Beecher (Sep 15)
- Re: Never push the Big Red Button (New York City subway failure) Fred Baker (Sep 15)
- Re: Never push the Big Red Button (New York City subway failure) Warren Kumari (Sep 15)
- Re: Never push the Big Red Button (New York City subway failure) William Herrin (Sep 15)
- Re: Never push the Big Red Button Stephen Satchell (Sep 15)
- Re: Never push the Big Red Button Keith Stokes (Sep 15)
- Re: Never push the Big Red Button Billy Croan (Sep 15)
- Re: Never push the Big Red Button Callan Banner (Sep 15)
- Re: Never push the Big Red Button Robert Taylor (Sep 16)
- Re: Never push the Big Red Button (New York City subway failure) Christopher Morrow (Sep 10)
- RE: Never push the Big Red Button (New York City subway failure) Matthew Huff (Sep 10)
- Re: Never push the Big Red Button (New York City subway failure) Daniel Seagraves (Sep 15)
- Re: Never push the Big Red Button (New York City subway failure) Warren Kumari (Sep 10)