nanog mailing list archives

Re: rack power question


From: Deepak Jain <deepak () ai net>
Date: Mon, 24 Mar 2008 23:27:14 -0400



While I enjoy hand waving as much as the next guy... reading over this thread, there are several definitions of sq ft (ft^2) here and folks are interchanging their uses whether aware of it or not.

1) sq ft = the amount of sq ft your cabinet/cage sits on.

2) sq ft = the amount of sq ft attributed to your cabinet/cage on the data center floor including aisles and access-ways

3) sq ft = the amount of sq ft attributed to your cabinet/cage on the data center floor including aisles and access-ways and on-the-floor cooling equipment

4) sq ft = the amount of sq ft attributed to your cabinet/cage on the data center floor including aisles and access-ways and on-the-floor cooling equipment AND the amount attributed to your cabinet/cage from the equipment room (UPS, batteries, transformers, etc).

The first definition only applies to those renting cabinets.
The first/second definitions apply to those renting cabinets and cages with aisles or access-ways in them The first/second/third definitions apply to operators of datacenters within non-datacenter buildings (where datacenter is NOT the entire load in the facility) and renters. All the definitions apply to anyone with a dedicated datacenter space (and equipment room) within a building or a stand-alone datacenter.

By rough figuring...

A 30KW cabinet while one sounds lovely, a huge amount of space is going to turned over to most or all of a dedicated PCU and 1/15th of the infrastructure of 500KVA UPS (@0.9PF) including batteries, transformers, etc.

Assuming power costs and associated maintenance are assigned appropriately to this one cabinet, the amount of square footage associated (definition #4) for that one cabinet changes by less than 30% whether you are going 30KW in one-cabinet or 3KW in each of 10 cabinets.

As an owner/operator of very large dedicated data centers for very large customers of all sorts, I can promise you no one is doing datacenters full (500+ cabinets) of 10KW+ (production, not theoretical) each in a dedicated facility with no other uses to lower the average heat demand. Even smaller numbers probably too.

Easy caveat:

A "datacenter" that is a fraction of a large building (e.g. a 20,000 sq ft data center within a 250,000 sq ft building) can appear to bend these rules because the overall load (by definition #4) is averaged against it.

There is simply no economic reason to do so (at scale) -- short of water cooling -- there is a fixed amount of space taken up per unit-ton of air cooling (medium-<air>-medium) for heat-rejection. Factor in the premiums associated with the highest density equipment (e.g. blades, PDUs -in-cabinet, etc) and the economics become even clearer.

Even ignoring heat rejection, the battery + UPS gear for 500KVA (even with minimal battery times) is approximately the same size (physically) as the 12 cabinets or so it takes to reach that capacity. [same applies for flywheel/kinetic systems]

Our friends who do calculus in their heads can already figure out the engineering or business min-max equation to optimize this equation based on a certain level of redundancy, run-time, etc and there aren't multiple answers. (Hint: certain variables drop out as rounding errors).

TAANSTAFL, if you are a 1-4 cabinet (or similarly small) use in a larger datacenter (definitions 1-2) by all means shove as much gear as you can in as long as there is no additional power premium. If they are giving you space for power or the premium is too high, take as much space as you can for the amount of power you need -- your equipment and your budgets will thank you. If you are operating a data center without a bigger use in the building to average against, you really don't have many ways to cheat the math here. (e.g. geothermal only provides a delta between definition #3 and #4 and a lower energy premium).

Deepak Jain
AiNET







Current thread: