Page 1 of 2

Plotting Overall Folding System Efficiency

Posted: Tue Jan 29, 2019 2:37 am
by gordonbb
I recently acquired a couple of Eaton dual conversion UPSes and Network Cards. I've managed to hook them up to a couple of folding Rigs and I have achieved Stats Nirvana.
GTX 1070Ti and RTX2070
Image
2 x GTX 1060 6GB
Image
I query the PPD from the FAHClient on the system and take that value and divide it by the UPS output wattage via SNMP from it's Network Interface and then divide by my average cost of $0.13Cdn per kW.
So the two higher end cards are much more efficient and I'm averaging about $1/day per system in Electrical usage.

Re: Plotting Overall Folding System Efficiency

Posted: Tue Jan 29, 2019 3:54 am
by ProDigit
Something appears to be off on the graph.

At $0.13/kWh
and assuming your systems are well set up,
the 1070 + 2070 will run at 1M PPD/card, and 450Watts (200Watts per card, ~50Watts for the system),
the 1060 + 1060 will run at 350k PPD/card, @ 250Watts (100Watts per card, ~50Watts for the system),

Your first system should consume about $1.4 / day,
Your second system about $0.78/day.

Your first system should run at around 1.4M PPD/$
Your second system should run at around 900k PPD/$

Minor variances in system are possible, but on average that's closer to a 1.5-1 ratio than the reported 2-1 ratio.
Most of the point differences are due to the bonus points on faster returns on the 1070/2070 system.
If bonus points were not counted, or finished WUs of same length were counted instead of PPDs, the ratio should be much closer to 1.2-1

Re: Plotting Overall Folding System Efficiency

Posted: Tue Jan 29, 2019 9:28 pm
by gordonbb
No, these are measured values. The PPD agree with the numbers reported by HFM.net and Stanford and the Power agrees with measurements from a Kill-a-Watt.

The 2 1060s average 951kPPD over a 14 day period and draw 118W each (120W Limit set) so the calculated system overhead is 61.5W.

The 1070Ti averages 910kPPD and the 1070 1.23MPPD over a 14 day period and draw 138 and 168W respectively (140 and 170W power limits) so the calculated system overhead is 44.5W.

I believe the vast difference in system overhead on the Rig with the 1060s is due to the capacitors needing to be replaced on the inverter board on the UPSes which are several years old. I have to replace the fan in. The other UPS so I’ve ordered 2 fans and two sets of electrolytic capacitors and will change them both out at the same time as they are both of the same vintage.

Re: Plotting Overall Folding System Efficiency

Posted: Tue Jan 29, 2019 9:43 pm
by ProDigit
odd. Most sites report the 1070 ti to run at 180Watts under load, and the 2070 at 220Watts.

My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.

Re: Plotting Overall Folding System Efficiency

Posted: Wed Jan 30, 2019 7:46 am
by rwh202
ProDigit wrote:odd. Most sites report the 1070 ti to run at 180Watts under load, and the 2070 at 220Watts.
Folding doesn't get the high end Nvidia GPUs near their TDP since it's not exercising everything on the card. Plus, most choose to set a lower power limit to operate more efficiently anyway.

I've got 1080 T's that don't get above 180W, despite 250W power limits and plenty of cooling.

Re: Plotting Overall Folding System Efficiency

Posted: Wed Jan 30, 2019 10:31 am
by Holdolin
While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.

Re: Plotting Overall Folding System Efficiency

Posted: Wed Jan 30, 2019 2:27 pm
by Mstenholm
I like it when somebody does make a little investigation into GPU power consumption during folding. I have been a bit lazy lately but what I found with a GTX 970 stock under Linux is that it uses 145 W (difference between idle and load) and that HWInfo report very close to real Kill-a-Watt meter under Windows for slightly OCed 1070, 1070ti and 2070. These values are 10 % lower then TPU average gaming.
I so envy your low kWh price.

Re: Plotting Overall Folding System Efficiency

Posted: Wed Jan 30, 2019 8:53 pm
by kbeeveer46
Holdolin wrote:While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.
If you download the GPU-Z app and click on the Sensors tab it will show you how much power the card is pulling. I'm folding right now and my 2070 is pulling 162 W.

Re: Plotting Overall Folding System Efficiency

Posted: Thu Jan 31, 2019 12:58 am
by gordonbb
Sites may report a typical power draw but I purposely power limit the cards.

What I found during FAHBench testing, which does not take into account the Quick Return Bonus, is there is a Knee at the top of the power vs. Performance curve. So I run my cards power limited to 20W or so below the maximum theoretical limit and observe only a slight degradation in performance.

As for the 1060s performance these are installed in a Fractal Define R4 case with 6 140mm PWM fans and custom Fan Curves to balance noise and airflow so they manage to run around 1950MHz with the GPUs in the mid 60s for the lower card and low 70s for the upper card. The GPU fans are set to 50% as that is what seems to balance temperatures best between the two cards. The default fan curves work OK but tend to force the upper card to run in the high 70s.

Re: Plotting Overall Folding System Efficiency

Posted: Thu Jan 31, 2019 1:02 am
by gordonbb
ProDigit wrote:... My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.
Check your temperatures and GPU clocks. I suspect GPU Boost is downclocking your card. Or Roy you have a 3GB version with less shaders?

Re: Plotting Overall Folding System Efficiency

Posted: Thu Jan 31, 2019 7:38 am
by rwh202
gordonbb wrote:
ProDigit wrote:... My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.
Check your temperatures and GPU clocks. I suspect GPU Boost is downclocking your card. Or Roy you have a 3GB version with less shaders?
Also linux vs windows.
I've a 6GB 1060 under linux doing 450k PPD, and a 3GB 1060 in windows doing 340k PPD. Don't know how much is the loss of shaders and how much is OS, but I suspect it's a bit of both.

Re: Plotting Overall Folding System Efficiency

Posted: Thu Jan 31, 2019 12:03 pm
by Nathan_P
I have a GTX 1070 and 1080 on a dual CPU Asus Z9PE with dual 12c/24t xeons and a 1070Ti and 1050Ti on a Z87 WS with an E3-1230L v3 currently pulling 742w from then wall for 2.316m PPD. Both systems run on Mint 18.2, clocks and voltages are whatever the cards are set to act the factory. I have seen power draw go as high as 840w but its all project dependent.

Re: Plotting Overall Folding System Efficiency

Posted: Thu Jan 31, 2019 11:06 pm
by ProDigit
kbeeveer46 wrote:
Holdolin wrote:While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.
If you download the GPU-Z app and click on the Sensors tab it will show you how much power the card is pulling. I'm folding right now and my 2070 is pulling 162 W.
This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.

Re: Plotting Overall Folding System Efficiency

Posted: Fri Feb 01, 2019 10:01 am
by rwh202
ProDigit wrote:This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.
I'd be surprised if risers affected anything (other than slowing the cards down and reducing power consumption)
The power sensors are on the card and don't care whether the power comes from the slot, the riser or 6/8-pin power connectors. However, I'm not sure if they are pre- or post-VRM and whether the losses there are included.
The difference between killawat and GPUZ is likely down to PSU efficiency.
107 + 63 = 170 W total for GPU. Assuming 85% PSU efficiency, then you have 200 W from the wall as shown by the killawat.

Re: Plotting Overall Folding System Efficiency

Posted: Fri Feb 01, 2019 10:51 am
by ProDigit
rwh202 wrote:
ProDigit wrote:This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.
I'd be surprised if risers affected anything (other than slowing the cards down and reducing power consumption)
The power sensors are on the card and don't care whether the power comes from the slot, the riser or 6/8-pin power connectors. However, I'm not sure if they are pre- or post-VRM and whether the losses there are included.
The difference between killawat and GPUZ is likely down to PSU efficiency.
107 + 63 = 170 W total for GPU. Assuming 85% PSU efficiency, then you have 200 W from the wall as shown by the killawat.
You are totally right. I forgot that the 1060 running on the 1x port is only operating at half the PPD, which might be the reason why the lower power draw.