[help] advice needed to build FAH rig

[ari2asem]

I wrote this post specifically for what you are looking to do: viewtopic.php?f=38&t=28869

This is my eighteenth year folding so I have a little experience. I have two solid machines running with the AMD FX-8350/8370 CPU's for years now. They are cheap to acquire. Then I use the Gigabyte 990FXA-UD3/UD5 motherboards. On the UD5 I am running 5 Nvidia GPU's. (1080Ti, 1080,1080,1070,1070) with a 1300w PS. You do want a faster CPU clock as even on Linux it still takes advantage of it. Hyper-threaded CPU you would want to turn off as it actually impacts folding if you use multiple GPU's. My power bill for all 11 GPU's and machines is about $220/month.

take a look at this build. https://m.imgur.com/a/7dfC8ym

this is the same idea as your build. this build is crunching WU's for gpugrid.net. and for WCG
specs of the hardware:
- asrock x399 taichi (mainboard)
- threadripper 1950x (16 core, 32 threads cpu)
- pcie riser cables x16 >>> x16 (brand is Li-heat) https://www.highflow.nl/hardware/videok ... black.html
- 4x gpu msi gtx 1070 itx 8GB

this next setup is half away. tomorrow i get my mainboard (asus p9x79-e ws) and cpu (xeon e5-2690 v2). gpu's are 5x msi gtx 1070 itx 8GB.
https://m.imgur.com/a/U5gP4tW.
actually this is meant for gpugrid.net (this becomes my 2nd rig for gpugrid).

for FAH i am building (or trying to build -:) ) a system with the same idea:
- mining frame
- pcie riser cables x16 >> x16
- psu 1200 watt corsair

but the question is:
- buying mainboard with pcie v2 or with pcie v3? i read here that when going lower than x4 pcie v2 will cause major performance drop

- buying 6 gtx 1060 6gb gpu cards? (used and second hand with much better ratio of GFlops / € )
or
- buying 4 rtx 2070 (with lower ration GFlops / € than gtx 1060)
or better setup 4x rtx 2070 or 6x gtx 1060

consideration 1.) building 6 gpu setup (gtx 1060) will cost more on mainboard and cpu then gpus (using nearly new mainboard/cpu with older gpu)
consideration 2.) building 4 gpu setup (rtx 2070) will cost more on gpus then mainboard and cpu (using nearly new gpu with older mainboard/cpu)

fact 1.) eletrical bill is no issue to watch for it. all my builds (FAH, gpugrid, WCG) will be sitted at my work. no pay for electricity -:)
fact 2.) whichever consideration is chosen, it will always be built on mining frame, with pcie riser cables x16 >> x16


so, your thougths about consideration 1 and 2 ?
i am struggling and doubteing too much

[MeeLee]

Threadripper is indeed an interesting processor. More interesting than any other processor on the market today.
However, motherboards with multi PCIE ports are insanely expensive.
And take my advice, stick with RTX cards. Don't go for GTX cards.
You'll need twice the amount of GTX 1060 cards to match the RTX 2060 in folding.
Not only do they consume more power, they run hotter (more AC is needed), take more place, and many motherboards have issues with many GPU setups.

And I don't know if it makes any sense connecting more than 6 GPUs anyway; as 6 RTX 2060 to 2080's pull about 1kW of power at the wall; fully optimized; 1200W stock.
You don't want to run a 1200W PSU at 1200W. It operates most efficiently at 50% load (600W), and if it's a PSU Gold, it'll operate best under 80% load (960W), which is what 6GPUs will do when they're power optimized.
That way, you don't fry the PSU, whenever there's an outage, and the system reboots, and FAH loads all the cards at stock power levels.

With Intel and Threadripper, you have at least 20 PCIE lanes at 3.0.
For 2060s and 2070s, you'll be ok with PCIE 3.0 1x speed slots.
Most motherboards offer you 3 full size slots (or less), operating at 16/8/4x; 1 or 2 m.2 slots at 4/2/1x; and 3x slots at 1x.
It's an and/or kind of thing.

RTX 2070s don't fold much faster (about 10% faster) than an RTX 2060, but cost nearly double the price; which is why I'd prefer the RTX 2060s.
If you can wait for a month, you can get 6x new RTX 2060s, for the price of 4x 2070s. A new 2060 (some call them 2060Ti, some 2060 Super) will probably be about as fast as a 2070 in folding (or perhaps even a bit faster).
Or, you can just buy up the older 2060s. Their price will go down a bit. But it'll take another month before the newer cards will be released.
The newer 2060s will reportedly have 256 memory bus, instead of 192.

I would seriously not buy a 1060. A 1060 folds at about 500k PPD, while a 2060 does double.

If electricity is no issue for you, I'd choose between the 2060s, or the 2080s; whatever you can afford. 2070s don't fold well for their price.
2060 : 1.04 MPPD $350
2070: 1.12-1.23 MPPD $500
2080: 1.40 MPPD $700

[foldy]

For 2060s and 2070s, you'll be ok with PCIE 3.0 1x speed slots.

But you must use Linux for pcie 3.0 x1 as Windows bottlenecks and needs pcie 3.0 x4 minimum

[MeeLee]

But you must use Linux for pcie 3.0 x1 as Windows bottlenecks and needs pcie 3.0 x4 minimum

I would really recommend anyone, who has a dedicated PC for folding, to run Linux.
I assume it still is about 10% faster, and more energy efficient; as well as doesn't get interrupted by automatic updates; and system reboots.

[ari2asem]

ok, ppl….my test rig with 6x gtx 1070 is online, just started.
mainboard: asus p9x79-e ws (with 2 PLX8747 pcie switches, 7x pcie-slots)
cpu: xeon e5-2690 v2 (10 core, hypererthreading 20 units)
gpu: msi and gigabyte gtx 1070 8gb, undervolted with msi afterburner (power limit 50%)
folding speed gpu's: between 48k and 92k PPD, depending on project
folding speed cpu: 3700 PPD
folding power: set to FULL

[MeeLee]

ok, ppl….my test rig with 6x gtx 1070 is online, just started.
mainboard: asus p9x79-e ws (with 2 PLX8747 pcie switches, 7x pcie-slots)
cpu: xeon e5-2690 v2 (10 core, hypererthreading 20 units)
gpu: msi and gigabyte gtx 1070 8gb, undervolted with msi afterburner (power limit 50%)
folding speed gpu's: between 48k and 92k PPD, depending on project
folding speed cpu: 3700 PPD
folding power: set to FULL

Happy finetuning!
Seems a bit on the low side for a 1070. My 1060 got 335-350k PPD per card.

I'd be careful with undervolting, and map your PPD score, with the power curve for a few days.
Sometimes cutting power too much, brings it below optimal values.
Your 1070 should be getting 500-750k PPD.
Did you add your own passkey?

[ari2asem]

no passkey yet...should that make more performance?

[MeeLee]

no passkey yet...should that make more performance?

Definitely! You should get about ~7x higher PPD with a passkey on a GTX 1070.
Most of your points are from the return bonus.

[bruce]

The passkey does not increase hardware performance but it does increase the points.
(Even though we commonly judge "performance" by the points, that's a fallacy. Bonus points don't change how many WUs you've computed or how complex your assignments have been.)

See the discussion of QRB bonus points https://foldingathome.org/support/faq/points/passkey/
https://foldingathome.org/faqs/points/b ... etermined/

[ari2asem]

after making passkey I get now around 3,2 million PPD, with no power limit. total power draw of the whole system from the wall is about 850-860 watt for 6 gtx1070. power draw of each card is around 90-95 watt.

with power limit set to 50% is totall power draw of the whole computer around 630-640 watt, folding at 2,8 million PPD. power draw of each card is around 70-75 watt.

ì think I will stick with 50% power limit setup

[Theodore]

Now combine it with a mild GPU and vram overclock.
It shouldn't affect power draw by much.
And you can regain at least .2 MPPD, bringing you just over 3M PPD.

Also, add a few Watts to the minimum. When setting power caps to the absolute minimum, the card's performance drops much. Adding just 1 to 2 watts, or 1 to 2% brings performance closer to it's optimal value.

[toTOW]

Do you use Windows or Linux ?

On Windows, my 1070 @ 2 GHz gets 620k PPD ...

Also, you could disable CPU folding as it probably reduce overall GPU production by a bigger value that it produce ...