nanog mailing list archives

Re: 165 Halsey recurring power issues


From: Joe Maimon <jmaimon () jmaimon com>
Date: Tue, 31 Oct 2023 22:48:08 -0400

The building itself got into the action and their goal was to make a top notch facility focusing on central patch panel fiber cross connects. They started with half of the 9th floor originally called MMR-2 and continued with multiple spaces each bigger as it was quite successful. No raised floors, properly positioned chillers, ample power, basic but standard and roomy cabinets, one time fee per cross connect (plus initial cabling and panel setup OTC) and they have been very succesfull by all appearances.

Staff reflected their initial goals and I have always interacted well with them.

Original mmr where each xcon was actualy pulled space to space was quite a sight with multiple cable conduits and trays running from the tops of the cabs to the ceiling, all full. New space adopted modern approaches and looked it.

Joe

Sean Donelan wrote:
ine tume
165 Halsey (and most of its tenant) data centers is an older facility. Data center practices have changed over the decades, and terminlogy wasn't standardized until recently.


The biggest FUBAR in telco and data centers is the difference between "redundancy" and "diversity."

Redundant A/B power feeds are often multiple cables from the same power source.

Diverse A/B power feeds are cables from different backup power sources (within limits). 1-utility, 2 battery strings or backup generators. Often routed through in same conduits/cable trays. But both may be out of service for scheduled maintenance and some kinds of faults.

Add a spare A/B power feed. Generally a N+1 backup power source and some additional power switching capability.

Fault tolerant A/B power. Everything from utility to rack is diverse and redudant (cables, conduit/cable trays, switching equipment and backup sources). Maintenance can be performed on one of the power feeds without affecting the other feeds. Does not include redundant utility feed (not redundant substation, utility).

Cost increase 2x, 5x, 10x

I haven't toured 165 Halsey for 10+ years, so I don't know its current state. It has multiple tenant data centers, so some may be better than others.


On Mon, 23 Oct 2023, Babak Pasdar wrote:
Hello,

I wanted to get some feedback as to what is considered standard A/B power setup when data centers sell redundant power. It has always been my understanding that A/B power means individually unique and preferably alternate path connections to disparate UPS units.

A few months ago, 165 Halsey took us down for several hours. They claimed that a UPS failed causing this issue. Our natural reaction was that we have A/B redundant power so a failed UPS on the A circuit should not take down the cabinet. Joe the facility manager claimed that industry standard A/B power means two circuits to the same UPS, which makes no sense to me.

They committed to move us to A/B power with redundant circuits to disparate UPS units. However, we had a multi-hour outage again in that site this weekend. At first glance it seems to be the same problem.

We have checked with all of our other data center providers who have confirmed A/B power is in fact individually unique connections to disparate UPS units. 165 Halsey's definition of what constitutes redundant power seems unique. Why would anyone pay extra for a second connection to the same UPS? However, I wanted to get feedback to see if I am taking crazy pills here 🙂

None-the-less, we have lost all confidence in this facility.

Best Regards,

Babak





Current thread: