Re: DHS letters for fuel and facility access

[Paul Nash <paul () nashnetworks ca>]

You just have to make sure that you test the right thing.

In a former life I was an electrical engineer. My first job was with a consulting engineering firm; out biggest 
customer was the biggest supermarket chain in South Africa.  One of my tasks was to travel to one of their stores each 
Saturday after closing (those were the days when they closed at noon on a Saturday until Monday morning) and test their 
stand generators.

The manager’s idea was usually to press the start button, check that the big diesel started, then shut down and go 
home.  My idea was to pull the main incoming breaker.  9 times out of 10 on first visit, the diesel would start, and 
then die as soon as the load kicked in because of carbon buildup in the cylinders.

After discussions with the supermarket management, they decided to (a) have all the diesels serviced ASAP, and (b) 
adopt my protocol of start diesel, wait for it to come under load, run for at least 30 minutes to get up to heat and 
clear the carbon deposits.

I use a similar technique for failover tests on servers, routers, firewalls — pull the power cord and see what happens, 
pull the incoming network and see what happens.

This was stymied by a recent network outage where the ISP network was up and running, connected back to their local PoP 
and thence to their backbone, but connectivity from that network to the critical servers was down.  So now we test 
end-to-end that the server is reachable, and let the network fail over if not.

        paul

> On Mar 18, 2020, at 11:56 AM, Karl Auer <kauer () biplane com au> wrote:
>
> An untested emergency system has to be regarded as a non-existent
> emergency system.
>
> No matter how painful it is to test, no matter how expensive it is to
> test, the pain and the expense are nothing compared to the pain and
> expense of having an actual emergency and discovering that the
> emergency system doesn't work...
>
> Multiplied by infinity if it costs lives.
>
> Regards, K.
>
> -- 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Karl Auer (kauer () biplane com au)
> http://www.biplane.com.au/kauer
> http://twitter.com/kauer389
>
> GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
> Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D