I am looking to start 2016 with a dedicated folding rig. This PC will be used for nothing but folding, so I have no cares about its capabilities beyond stability and folding performance. Here is what I have put together as an initial configuration, and any thoughts/critiques from other folders are welcome. My selection of the CPU and motherboard were mainly driven by the requirement to fit 3 GTX 970 cards, and this is also why I opted for a 1000W PSU. Let me know if this is overkill/underkill/totally wrong. It's not the most expensive folding rig out there, but it is still a lot of money, so if I am throwing money away somewhere in there I'd like to know.
Also, I have some spare HDDs laying around which is why I am not buying one. The plan is to run Ubuntu as well...are there any known driver issues with it and the GTX 970 (or 3 of them)?
With this motherboard, it looks like with 3 GPUs the PCIe slots will be configured as 8X / 4X / 4X. How much of an impact does the number of lanes have on GPU folding performance?
The PSU is slightly over kill as a 970 usually only pulls about 145watts under a folding load, you could look at the 850w version but I'm not sure if it would have enough PCIe cables. PCIe - everyone claims something different, I've noticed about a 10% drop running in a PCIe x8 slot so a forum seach on that one would give you an average. Fan wise, unless you want asthetics, just get some cheap back fans from a name brand - no need to spend $60 on fans when you can do the same job for $30-$40
My only other thought would be maybe go for a pair of 980Ti's instead - you will get more PPD but I don't know how constrained you are on the budget.
Right, I was debating going with a 980 or 980Ti versus 2 970's. I think that one of those will yield as many PPD as a pair of 970's, and I would then have more slots available for adding more GPUs later on. My limit is about $1500, so I could do a 980Ti and a 970, and then add something else later on.
I will do some reading on the PCIe lane count and effects on PPD. Maybe it would in fact make more sense (from a future-proofing perspective) to find a motherboard+chipset that supports multiple slots at X16. I imagine that I would be looking at an additional $200-300 in motherboard & CPU costs in that case since I will probably need to go with a Z170 chipset and Skylake CPU.
A 980ti and a 970 will give you about 650k to 750k PPD depending on WU and how hard you overclock the card, I can't really advise on mobo/chipset combo's as I don't really keep up with the current stuff, all I would suggest is maybe look at x79/x99 or even low end server gear - you can get cheap xeon cpu's and they usually support upto 40 lanes - you don't even need to go current gen, sandy or ivy bridge based i7's or xeons would work just as well as haswell/skylake.
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Thanks! I was just finding that I would need an X-series chipset to realize 40 lanes. X99 boards are a little pricey, and holy moly the LGA2011-V3 CPUs are $$$$! I will look into older server equipment with X79 chipsets since the CPU is largely unimportant. Anyway, should I go with the assumption that having more lanes available to the GPU will yield improvements in WU throughput? I am not super familiar with the finer points of GPU architecture...why do the number of lanes matter when it sounds like the GPU has its own logic units and RAM onboard? The lanes are its link to the CPU, no?
From what I have read, CPU folding still yields some results, but it sounds like its days may be numbered and much of what is being done with the CPU is more or less checking the results from GPU projects. Is this a reasonably correct understanding?
If you go for a server based option, you can save $$$'s on the cpu, Sandy bridge xeons are going for about £150 now and newer ivy & haswell chips can be bought cheaply if you know which ES chips are good to use, personally I'd look at lower power ivy bridge - much better on the power consumption than sandy.
At least based on prices for new stuff, I think that X99 + Haswell Xeon makes sense.
Any glaring issues with using this mobo / CPU / RAM?
Haswell Xeon E5-2603 v3 (+$70)
MSI X99S (+$110)
2x4GB DDR4-2133 (+$15)
CPU cooler of some sort to handle 85W (+$60)
Total cost increase: $255
So, basically in doing this I would be able to run 3 GPUs at X16/X16/X8 (versus X8/X4/X4), and I would either buy 2 GTX 970 cards or 1 GTX 980/980Ti to start. Then down the road I could add more GPUs, up to the max of 3. Maybe I do want to stick with the 1000W PSU if I am thinking about the 980!
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Nathan_P wrote:The PSU is slightly over kill as a 970 usually only pulls about 145watts under a folding load, you could look at the 850w version but I'm not sure if it would have enough PCIe cables. PCIe - everyone claims something different, I've noticed about a 10% drop running in a PCIe x8 slot so a forum seach on that one would give you an average. Fan wise, unless you want asthetics, just get some cheap back fans from a name brand - no need to spend $60 on fans when you can do the same job for $30-$40
My only other thought would be maybe go for a pair of 980Ti's instead - you will get more PPD but I don't know how constrained you are on the budget.
I would also suggestion 980Ti's. I see a Zotac on Newegg for $599. I have a 970 and 980Ti and the 980Ti is more than double the PPD than the 970. You'd spend more for 2x 970s than 1 980Ti and get less PPD. The rest of the hardware is going to cost you the same, minus the PSU so might as well go big and maximize those PCI-E slots.
Depending on the PCI-E slot setup on the motherboard you may want to think about rear exhaust setups instead of the internal exhaust setups just because of how close they can be.
And agreed on the exhausting setup. It will be important for sure. On stock clocks things may be fine...but it sounds like some GPU overclocking can improve WU throughput so I may play around with that as well. My main beef with the rear-exhausting GPUs has been the noise level. The smaller diameter centrifugal blowers make a lot more noise than 2-3 axial fans. If I end up with axial fans, I may build some plenums to direct the air out the sides or back of the case. Noise won't be a huge issue since the PC will not be anywhere where people are regularly occupying, so I will keep my options open.
Whoa, forgot about Pascal. While I do want to get up and running well before that drops, I sure wish it was coming sooner! Such is life when it comes to "the next big thing" in PC hardware though.
bmwman91 wrote:Anyway, should I go with the assumption that having more lanes available to the GPU will yield improvements in WU throughput? I am not super familiar with the finer points of GPU architecture...why do the number of lanes matter when it sounds like the GPU has its own logic units and RAM onboard? The lanes are its link to the CPU, no?
More lanes will yield some improvements in WU throughput just like a faster disk will improve CPU throughput -- some (small?) percentage of the time the required data is not in local (V)RAM and it has to be fetched from a slower source.
Yes, the GPU has it's own VRAM onboard and plenty of superfast logic units, but (A) the VRAM doesn't hold enough data so it has to be paged to/from main RAM. The CPU manages the data movement plus some relatively minor processing. It also it writes checkpoints to permanent storage (i.e.-disk) every few minutes. Then, too, the onboard VRAM is pretty slow when you're talking about pretty large blocks of data.
Thank you Bruce, that makes sense. Does the amount of system RAM matter then, if the VRAM fill sup and the WU needs to page stuff into the system RAM? From reading around on here it sounds like the answer is no, but I'd like to make sure that I am setting this thing up properly the first time around!
The CPU I am looking to go with (E5-2603 V3) is 6 cores but only 1.6GHz. This means that it may take a bit longer to compile a newly downloaded WU, but other than that it will have no real material impact on WU throughput, correct?
