Guidance for building a folding dedicated computer

[ComputerGenie]

people tell you to just open a window to air cool 4 X gpu's they haven't run this

but i do
...

But, you're not exactly running a "stock" case in a "normal" room, either.

[boristsybin]

yes, the case is a bit customized (originaly Antec NSK 4482, very compact and spacy, AYCS), and the room... is watercloset, hot air blows directly into exhaust ventilation. No noise and no heat in livingrooms

3x1080Ti FE run their coolers up to 80% and temperatures up to 80 C

[Aurum]

Boris, Those are the biggest gauge PCIe power cables I've ever seen
Thanks for all the photos in your gallery. You must've had a lot of fun figuring all that out.

[boristsybin]

Boris, Those are the biggest gauge PCIe power cables I've ever seen

Yeah, I pimp my wires!

Thanks for all the photos in your gallery. You must've had a lot of fun figuring all that out.

Thank you.

[Leonardo]

I award extra points for the heat exhaust ingenuity!

[FldngForGrandparents]

7 slots doesn't necessarily mean it's capable of handling 7 GPUs... Since the CPU will handle a max of 40 PCI lanes, 5 x8 GPU connections would be the max available, IF you could physically put the cards into the machine. That board will accept a max of 4 double-slot GPU cards.

This is not correct. Everyone on here seems to think you need x8 PCIe. This depends on several factors. One being the GPU itself. I am currently running a 1070 on a PCIe 1x to 16x adapter connected to a PCIe 1x Gen3 slot and doing 700k PPD+. This has more to do with the PCIe gen when using multiple GPU's. And yes this does require extension cables to accomplish.

[boristsybin]

I am currently running a 1070 on a PCIe 1x to 16x adapter connected to a PCIe 1x Gen3 slot and doing 700k PPD+.

That is very good numbers. What FAH Client and driver version you use? Seems that 1060 is very suitible for x1 Gen2 on Linux

[bruce]

The characteristics of projects so vary. Small projects a different that big projects and there are probably variations in the particular type of analysis being done.

The "Everyone on here seems to think you need x8 PCIe" comes from questions about when the bus speed "matters" The subtlety here is that to some people, it "matters" if the only GPU isn't running at a constant 100% and to others, running two GPUs at, say, 95% is actually a big improvement.

Actually, a very fast GPU needs more bandwidth than a moderate or slow GPU -- especially if you demand that the GPU utilization always be 100%.
Slower GPUs work find with a lower bandwidth, and if you need to allocate lower bus bandwidth in order to accommodate more GPUs, any GPU will get more work done with an X1 connection that if it's not connected.

Only you can decide the best way to spread the resources of your PCIe lanes across whatever group of GPUs you can connect.

[bruce]

That is very good numbers. What FAH Client and driver version you use? Seems that 1060 is very suitible for x1 Gen2 on Linux

It's unlikely that either the client version of the driver version have very much to do with it.

If you want every project to run at 100%, then give your GPU unlimited bandwidth. If you'll settle for some running at 99% and others at 95%, then you've still made a good change.

Depending on the the MB, the total bandwidth that is allocated has a cap as well as the particular slot. Each device gets an allocation of a specific chunk at boot time. In some cases, adding a 1x slot can reduce another slot from 16x to 8x or something like that -- if that matters on the particular project and GPU that is involved.

[foldy]

The important part is the OS: 8x pcie speed recommendation for full speed of fast GPUs is for Windows only. Linux needs much less pcie bandwidth.

[Nathan_P]

The important part is the OS: 8x pcie speed recommendation for full speed of fast GPUs is for Windows only. Linux needs much less pcie bandwidth.

It may need less but you still see a performance drop off from anything lower than PCIe2.0 x8 on Linux

[bruce]

It may need less but you still see a performance drop off from anything lower than PCIe2.0 x8 on Linux

True, but it's still a personal judgement whether the drop-off is significant or not ... as compared to the addition of another GPU.

[foldy]

Who is interested in pcie bandwidth limits, we have this thread
viewtopic.php?f=38&t=28847

[ollie1983]

As I have posted in the newbie section, I am looking to build or buy a fairly brutish PC to fold with. It might get some VR gaming as well eventually.

I am thinking along the lines of an X99 platform (probably Asus Sabertooth), i7 6850K CPU and probably a GTX 1080ti slotted on.

Am I going along the right lines for serious folding output with long term reliability? OR should I be choosing a different processor/mobo and SLI or something? Are AMD CPUs/platforms worth using instead? Various reviews are saying AMD have now got the edge in various benchmarks etc etc, but folding is a distinctly different application.

Any suggestion of a sensible case with good potential airflow, CPU cooler and PSU would also be useful.

I have built PCs before but my current rig is a few years old now and I just keep skipping along GPU generations to keep it up fairly current. I am just not that up to date with my knowledge of the respective hardware families these days.

Thanks

[boristsybin]

As I have posted in the newbie section, I am looking to build or buy a fairly brutish PC to fold with. It might get some VR gaming as well eventually.

Asus WS-series motherboard
Intel 8-thread CPU
16 to 32 Gb RAM
4 to 7 1080Ti
PSU by Seasonic / Corsair / EVGA / Superflower with common +12v line 1200 W and more
watercooling