nanog mailing list archives
Re: San Francisco Power Outage
From: Jeff Aitken <jaitken () aitken com>
Date: Wed, 25 Jul 2007 08:44:19 -0400
On Tue, Jul 24, 2007 at 09:57:09PM -0500, Brandon Galbraith wrote:
It appears that 365 is using the Hytec Continuous Power System [ http://hitec.pageprocessor.nl/p3.php?RubriekID=2016], which is a motor, generator, flywheel, clutch, and Diesel engine all on the same shaft. They don't use batteries.
Yes. I used to work for the company that originally built the 365 Main datacenter and remember touring it near the end of the construction phase. The collection of power units up on the roof was impressive, as were the seismic isolators in the basement. But even when you try and do everythying right Murphy usually finds a way to sneak up behind you and whisper "BOHICA" in your ear. For example, we had a failure at another datacenter that uses Piller units, which operate on the same basic principle as the Hitec ones. While running on generator one of the engines overheated due to an oil-flow problem and threw a rod. When the on-duty electrician responded to the alarm, there were red-hot chunks of engine *outside* of the enclosure, and there was a hole in the side of the unit large enough to stick your arm in. The facility manager kept the damaged piston as a momento. :-) I don't remember whether this was due to a design flaw, improper installation, or what, but the important points are that (1) this is the real world and shit happens, and (2) it wasn't until the generator was worked long enough that the reduction in oil flow caused enough friction to trigger a catastrophic failure. I.e., there's no guarantee that you will catch this kind of problem in your monthly tests. On Tue, Jul 24, 2007 at 05:39:34PM -0700, George William Herbert wrote:
Unfortunate real-world lesson: there is a functional difference between pushing the UPS test cutover button, and some of the stuff that can happen out on the power lines (including rapid voltage swings, harmonics, etc).
Precisely. --Jeff
Current thread:
- Re: San Francisco Power Outage, (continued)
- Re: San Francisco Power Outage Justin M. Streiner (Jul 24)
- Re: San Francisco Power Outage Michael Painter (Jul 25)
- Re: San Francisco Power Outage Patrick Giagnocavo (Jul 24)
- Re: San Francisco Power Outage Jim Popovitch (Jul 24)
- RE: San Francisco Power Outage Alex Rubenstein (Jul 24)
- RE: San Francisco Power Outage Raymond L. Corbin (Jul 24)
- Re: San Francisco Power Outage Owen DeLong (Jul 24)
- Re: San Francisco Power Outage Brandon Galbraith (Jul 24)
- RE: San Francisco Power Outage Randy Epstein (Jul 25)
- RE: San Francisco Power Outage Brian Wallingford (Jul 25)
- Re: San Francisco Power Outage Jeff Aitken (Jul 25)
- Re: San Francisco Power Outage Paul Vixie (Jul 25)
- Re: San Francisco Power Outage Jason Matthews (Jul 24)
- Re: San Francisco Power Outage Michael Loftis (Jul 24)
- Re: San Francisco Power Outage Jonathan Lassoff (Jul 24)
- Re: San Francisco Power Outage Jason Matthews (Jul 25)
- Re: San Francisco Power Outage Paul Vixie (Jul 25)
- Re: San Francisco Power Outage Tuc at T-B-O-H.NET (Jul 24)
- Re: San Francisco Power Outage Stephen Wilcox (Jul 25)