Page 1 of 1

Building a computer with several GPUs

Posted: Tue Feb 28, 2023 11:04 pm
by nchowlett
Heya Folding mods (and users)!

I've done some work building and configuring a computer that has several GPUs dedicated to Folding@home, which are in parallel use.

From my perspective it wasn't too hard, just needed to spend most time sourcing cost-effective GPUs & some adapting hardware. I've written an article giving guidelines for those interested in building something similar, which can be read online: https://www.tessellate.science/microsco ... rcomputer/

Note I did had issues with FAHClient 7.6.21 so installed fah-client 8.1.13 instead (which completed it's install without any errors on Ubuntu 22 LTS). I also had to revert to a slightly older kernel to allow Ubuntu to boot in the first place.

Regards, Nick

Re: Building a computer with several GPUs

Posted: Wed Mar 08, 2023 12:43 am
by BobWilliams757
Overall a good quick breakdown of information for someone wanting to contribute to folding. Over the years a number have done blogs and such, and the exposure certainly can't hurt anything. I know that one of the other forums members has done a number of projects on his systems on his site, https://greenfoldingathome.com/

But the link you had to the LTT thread has similar information on power levels and efficiency. He brings up power supply efficiency and others as well, but in the end any information helps.


Just curious, what did you end up using for your rig?

Re: Building a computer with several GPUs

Posted: Mon Mar 13, 2023 5:07 am
by nchowlett
Thanks Bob. I used a HP z420 motherboard (that attached to a mining rig chassis without issue) with 4 x RTX 3060 Ti's attached using all available PCIE slots, powered with a EVGA 750 (Watt) GA supply. The power supply is under powered, but underclocking the GPU's has resulted in no apparent shutdowns so far!

Re: Building a computer with several GPUs

Posted: Tue Jun 13, 2023 12:58 am
by nchowlett
Heads up I've run into issues with using an underpowered power-supply (I think), which I've manually got around via opening the window to cool the room down (which I believe is allowing the GPUs to run cooler and use less power), then quickly entering the command line and manually modifying power usage using code within the link I provided before.

Either that or when I open a browser power consumption exceeds the PSU). I'm trying to develop some automated script to do this on boot (my first one Ubuntu ignored for some reason), via throttling GPU usage before it enters the OS.

Re: Building a computer with several GPUs

Posted: Tue Jun 13, 2023 1:41 am
by bikeaddict
PSUs typically have some headroom (20-30%) above their rated capacity before they will trip the protection. You should be able to output 900W+ if it's working properly.

My GPUs are set with a power limit using nvidia-smi -pl [limit] rather than limiting the clock speed. The GPU will boost to as high a clock as it's capable while not exceeding the set power limit.

You can see how much power a GPU is drawing with nvidia-smi dmon or watch -n1 nvidia-smi for realtime monitoring. And you can use a device like a Kill A Watt or Gardner Bender power meter to see how much the PSU is drawing from the wall combined with the 80 Plus rating to estimate the PSU's actual output.

I use a Kasa/TP-Link KP115 energy monitoring smart plug to monitor power use on my setup.

Re: Building a computer with several GPUs

Posted: Mon Aug 07, 2023 11:17 pm
by nchowlett
Thanks for the power details! Using my GPUs at full throttle might actually come very close to 1000W (in addition to CPU power), which might exceed even the headroom.

Either way, something weird is going on as my system is randomly stalling after weeks of folding consistently: physically looks like its working (fans spinning and lights), but cannot access OS anymore. Currently I just reboot when noticed, but looking at scripting a checker that emails me when this occurs.

Re: Building a computer with several GPUs

Posted: Tue Aug 22, 2023 1:45 am
by nchowlett
Think I know what causes the stalling of folding. Our internet drops out intermittently (which I believe caused my OS's connection software to die), thus killing the request that asks for work from F@h's end.

Re: Building a computer with several GPUs

Posted: Thu Sep 28, 2023 12:49 am
by nchowlett
nchowlett wrote: Tue Aug 22, 2023 1:45 am Think I know what causes the stalling of folding. Our internet drops out intermittently (which I believe caused my OS's connection software to die), thus killing the request that asks for work from F@h's end.
Turns out this stalling was from F@h's software, it's been addressed and I've had no issues.