Re: DPDK and energy efficiency

[Eric Kuhnke <eric.kuhnke () gmail com>]

> Server PS maximum input wattage is 900W.  Present draw of 2.0A @ 208V is
~420W, so 420/900 = 46.67%

But in the real world an R640 would *never* draw 900W. Even if you were to
load it up with the maximal CPU configuration (2 x 125W TDP CPU per
socket), a full load of 2.5" 15K spinning drives, maximum RAM, and three
high wattage low-profile PCI-E cards, while simultaneously running CPU, RAM
and disk stress tests, you might get in the neighborhood of 550-600W under
load.

Much the same way that a desktop PC equipped with a nominally "850W" rated
active PFC 80+ gold power supply might be powering a motherboard and CPU
combo with a high CPU TDP, but total power consumption under stress
tests/benchmarks would be nowhere near 850W. That rating exists to ensure
that the power supply isn't running anywhere near its max capacity...



On Fri, Mar 5, 2021 at 3:33 PM Brian Knight <ml () knight-networks com> wrote:

> On 2021-03-05 15:40, Eric Kuhnke wrote:
>
> > For comparison purposes, I'm curious about the difference in wattage
> > results between:
> >
> > a) Your R640 at 420W running DPDK
> >
> > b) The same R640 hardware temporarily booted from a Ubuntu server live
> > USB, in which some common CPU stress and memory disk/IO benchmarks are
> > being run to intentionally load the system to 100% to characterize its
> > absolute maximum AC load wattage.
>
> We've got a few more hosts waiting to be deployed that are configured
> almost identically.  I'll see what we can do.
>
> I'm guessing those tests would pull slightly more power than the vEdge
> hosts, just because there's not much disk IO that happens on a
> networking VM.  These hosts have four SSDs for local storage.
>
> > What's the delta between the 420W and absolute maximum load the server
> > is capable of pulling on the 208VAC side?
> >
> > https://manpages.ubuntu.com/manpages/artful/man1/stress-ng.1.html
>
> Server PS maximum input wattage is 900W.  Present draw of 2.0A @ 208V is
> ~420W, so 420/900 = 46.67%
>
> > One possible factor is whether ESXI is configured to pass the pci-e
> > devices directly through to the guest VM, or if there is any
> > abstraction in between. For non-ESXI stuff, in the world of Xen or KVM
> > there's many different ways that a guest domU can access a dom0's
> > network devices, some of which can have impact on overall steady-state
> > wattage consumed by the system.
>
> The 420W server has its interfaces routed through the ESXI kernel.
> We're moving quickly to SR-IOV on new servers.
>
> > If the greatest possible efficiency is desired for a number of 1U
> > things, one thing to look at would be something similar to the open
> > compute platform single centralized AC to DC power units, and servers
> > that don't each have their own discrete 110-240VAC single or dual power
> > supplies. In terms of cubic meters of air moved per hour vs wattage,
> > the fans found in 1U servers are really quite inefficient. As a
> > randomly chosen example of 12VDC 40mm (1U server height) fan:
> >
> > https://www.shoppui.com/documents/9HV0412P3K001.pdf
> >
> > If you have a single 12.0VDC fan that's a maximum load of 1.52A, that's
> > a possible load of up to 18.24W for just *one* 40mm height fan. And
> > your typical high speed dual socket 1U server may have up to eight or
> > ten of those, in the typical front to back wind tunnel configuration.
> > Normally fans won't be running at full speed, so each one won't be a
> > 18W load, but more like 10-12W per fan is totally normal. Plus two at
> > least two more fans in both hot swap power supplies. Under heavy load I
> > would not be surprised at all to say that 80W to 90W of your R640's
> > total 420W load is ventilation.
>
> Which is of course dependent on the environmentals.  Fan speeds on our
> two servers are 25% for the 260W vs. 29% for 420W, so not much
> difference.  Inlet temp on both is ~17C.
>
> I checked out another R640 heavily loaded with vEdge VMs, and it's
> pulling similar power, 415W, but the fan speed is at 45%, because inlet
> temp is 22C.
>
> The TDP for the Xeon 6152 is 140W, which seems middle-of-the-road.  From
> the quick survey I did of Dell's configurator, the R640 can take CPUs up
> to 205W.  So we have headroom in terms of cooling.
>
> > In a situation where you're running out of power before you run out of
> > rack space, look at some 1.5U and 2U high chassist that use 60mm height
> > fans, which are much more efficient in ratio of air moved per time
> > period vs watts.
>
> Or ask the colo to turn the A/C lower ;)  (that moves the power problem
> elsewhere, I know)
>
> Thanks,
>
> -Brian