Many GPUs on one machine

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
nchowlett
Posts: 48
Joined: Tue Feb 28, 2023 4:05 am
Hardware configuration: Modified 'mining' rig with 4 parallel GPUs. For more info check out this article: https://tessellate.science/microscoped- ... ercomputer
Location: Melbourne, Australia.

Re: Many GPUs on one machine

Post by nchowlett »

Peter_Hucker wrote: Thu Sep 28, 2023 10:59 pm My GPUs aren't even inside the cases, I use mining risers on USB cables.
Similar as my current setup! But I'm looking at custom-designing a wooden case with metal plaque with some textual info. about F@h written on it - geared towards marketing/awareness of the whole folding endeavour.

Q: Do you think the brand of vendor (MSI, Gigabyte, etc) has an affect on the cards longevity? My other MSI card has gone offline.
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

I've used every vendor, and have never noticed one being better than others. It's like cars, they all wear out equally. They may have slightly different designs for cooling etc, but they're very similar.

I saw a strange card recently, the RAM wasn't joined to the heatsink, just relying on airflow. It was an R9 280X. Every other card using that chip had the RAM connected to the main heatsink, this had nothing at all. But it's just as reliable. Maybe they used ultra low power RAM? Maybe they decided the heatsink was feeding heat to the RAM from the GPU?
nchowlett
Posts: 48
Joined: Tue Feb 28, 2023 4:05 am
Hardware configuration: Modified 'mining' rig with 4 parallel GPUs. For more info check out this article: https://tessellate.science/microscoped- ... ercomputer
Location: Melbourne, Australia.

Re: Many GPUs on one machine

Post by nchowlett »

Maybe they decided the heatsink was feeding heat to the RAM from the GPU?
Maybe they realised that fancy new (i.e. insignificant) 'features' sells cards?
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

How would it be a selling feature? "This card comes supplied with non-cooled RAM!"
BobWilliams757
Posts: 521
Joined: Fri Apr 03, 2020 2:22 pm
Hardware configuration: ASRock X370M PRO4
Ryzen 2400G APU
16 GB DDR4-3200
MSI GTX 1660 Super Gaming X

Re: Many GPUs on one machine

Post by BobWilliams757 »

People fall for all kinds of flashy stuff that serves no purpose. There is probably a lot of stuff computer related that falls into that category. Does that heat sink actually help cooling or not? Short of testing, most of us will never know, but it LOOKS like it might help cooling, so people buy it.
Fold them if you get them!
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

I have 12 different Tahiti (280X) cards by different makes. About 5 different heatsink arrangements. Direct CUII for example. Every single one achieves the same temperature. The only difference is some run at a slightly different clock speed (not sure how some can do it and some don't, they're all 100% reliable, and they all use the same GPU chip, or do some get a higher bin level?). Also the RAM clock differs, but that may be a different set of chips. And some blast heat to the back and front, and some to the top and bottom. No direction is perfect. All to the back means more noise. Any other direction heats up the inside of the case. And down blasts the motherboard directly which I think is a bad idea. I'd choose back and front, which is what most but Gigabyte choose. Doesn't matter for me as they're on a shelf beside the computers connected with USB risers. I don't particularly like whatever heatsink paste they use, when I replace it they run 20C cooler. Maybe it wears out? Maybe they use cheap stuff? I do spend £10 a card (not much compared to the original price of £300) to change it to 17W/mK gunk (I don't use pads, they don't squish so much, so you don't get the GPU and RAM as close as possible).
BobWilliams757
Posts: 521
Joined: Fri Apr 03, 2020 2:22 pm
Hardware configuration: ASRock X370M PRO4
Ryzen 2400G APU
16 GB DDR4-3200
MSI GTX 1660 Super Gaming X

Re: Many GPUs on one machine

Post by BobWilliams757 »

I'd hate to see that power bill vs points comparision! Yikes!


But your example makes my point. All those different cooling solutions end up at about the same temp, maybe slightly different clock speeds means some of them cool a little bit better, but not much. But there are probably a lot of cases where without having that data, people buy Brand X because of the "ZOMG Suuuuper Cooooler" solution they have.
Fold them if you get them!
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

I buy by specs. Price vs clock speed. The pretty colours don't sway me.
Power can be cheap if you pick your source.
12 year old GPUs can be very cheap.
nchowlett
Posts: 48
Joined: Tue Feb 28, 2023 4:05 am
Hardware configuration: Modified 'mining' rig with 4 parallel GPUs. For more info check out this article: https://tessellate.science/microscoped- ... ercomputer
Location: Melbourne, Australia.

Re: Many GPUs on one machine

Post by nchowlett »

12 year old GPUs can be very cheap.
Sure, upfront. But the electricity costs are relatively big compared to newer cards. The efficiency gains on power usage over the last ~10 years are significant (approx. 10 x core count for same power draw). Also, NVIDIA RTX 4 series have better folding performance than 3 series for lower core count [1] (and lower price for 4 series when released). But buying used cards might nullify this difference.

[1] Comparing RTX 4070 to 3080: https://folding.lar.systems/gpu_ppd/brands/nvidia/
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

Like I said, electricity can be very cheap or even free after some initial outlay. Don't follow the rest of the world and just buy it from the grid, make your own! 90% cheaper! And once it's there, you might aswell use all of it.

It also provides heating, so if you're heating the house anyway, does it matter if the GPU is inefficient?

Also I treat old tech like pets, killing them off is just wrong. Work them into the ground! I've even got a dual core CPU here I bought for £1.25. Bit silly really as it cost the seller £1.55 to post it.
thiblec71
Posts: 21
Joined: Wed Jul 26, 2023 1:42 pm

Re: Many GPUs on one machine

Post by thiblec71 »

Dear all,
I already have a rtx4090 running 24/7 on a desktop gaming style case. I would like to build/buy a case to put 4 rtx 4090 at work solely for F@H. I run Ubuntu, onlt F@H and the card consume 350w of power.
Want setup, build and parts would you recommend?
thanks
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

Not sure any case or motherboard takes 4 cards unless you're talking about removing the aircoolers and watercooling them. I connect mine with USB mining risers so I can sit them next to the case.
thiblec71
Posts: 21
Joined: Wed Jul 26, 2023 1:42 pm

Re: Many GPUs on one machine

Post by thiblec71 »

Would you have a picture of your setup?
Also could you list the parts you are using for the configuration?
Peter_Hucker
Posts: 308
Joined: Wed Feb 16, 2022 1:18 am

Re: Many GPUs on one machine

Post by Peter_Hucker »

My setup is nothing special, just a number of basic tower cases with GPUs sat on the shelf next to them.
I use this sort of thing to connect them: https://www.ebay.co.uk/itm/145392436570 or https://www.ebay.co.uk/itm/333915444816 (those are the first I found, not necessarily the cheapest). For folding with decent speed GPUs, it's best to give each GPU it's own PCI socket and not share using the quad adapters, or there isn't enough bandwidth to keep them busy. For Boinc or slower GPUs, you can share sockets. A CPU with fast processing per core, and one core per GPU, is best as Folding only uses one core to feed each GPU.
thiblec71
Posts: 21
Joined: Wed Jul 26, 2023 1:42 pm

Re: Many GPUs on one machine

Post by thiblec71 »

I thought that USB PCI riser were limiting the performance of the GPU?
Post Reply