Page 1 of 3

Open Air Dedicated Folder and Guide

Posted: Wed Jun 01, 2016 7:38 am
by FldngForGrandparents
I finally have a little time to post my dedicated folding server. This is a custom open air case built for $20 from wood at Home Depot. It is very easy to do. It is very solid and can handle up to 6 GPU's with the right motherboard combo. I am currently using the Gigabyte 990FXA-UD5 and AMD FX-8350 with no overclock. I am only folding on the GPU's.

Image

These are 3/4" square 3ft long wood dowels found at Home Depot/Lowes and many other places. I used some nicer 1-1/2in white washer-head #8 cabinet screws that have the star driver with them. Be sure and screw them in so the head is flush and not pushing into the wood. They look nicer and clean.

Image

The case is based off the Litecoin/Dogecoin design (Link below). I made some minor adjustments to give a little more room for the power supplies to sit. But the base configuration will work just fine. You can attach two power supplies to this setup if you need.

Image
(Back View Above)

Motherboard and SSD attached to case. Be sure and align the motherboard so the PCIe slots are in the middle of the case as much as possible.
I used the standard motherboard case screws (gold posts) to hold the motherboard in place. I drilled a small hole and then screwed them in. Next time I would use a little gorilla glue to help hold them in place.


Image
(Front View)

Image

The two GTX 980 and Asus Strix 980 Ti video cards attached to the case. I am still testing the PCIe 1x to 16 adapter that you can see connected to the left card. So far I am seeing only minor PPD differences.

For the two additional GPU's I am using the EZDIY PCIe 16x extension cables (Black cable shown above and below). These are some of the best cables I have seen. Good solid solders. Completely shielded. And both ends have locking connectors. They are about 11 inches long which provides ample length to reach most of your GPUs.

Image


Image

To do more than one power supply you can use the Add2PSU Multiple Power Supply Adapter (Displayed Above). This connects the two power supplies in a way so when you power on/off the primary supply it will power on/off the secondary. This can be helpful with multi GPU systems and save some money as two 850 watt power supplies can be cheaper than a single 1200watt or even 1600 watt supply.

Image

I am using a EVGA 1000GQ 1000watt 80+ Gold power supply. It can be found on Amazon for $130. This is a modular power supply that can handle, and has cables, for four GPUs. I am currently pulling 720watts from the wall at max output.

You can see there still is space for a second power supply. You can also increase the size of the open air case when you build it if you need more space.



Image

And here is the system in action for the last month. The GPUs are running at around 52 celsius with fan speeds of 65% on the GTX 980 Ti and 70% on the GTX 980 reference cards. I am currently doing on average 1.45 million PPD. This is a mix of core 18 and 21. With all core 21 for the day I have seen upwards of 1.7 million PPD. From what I have read I can probably do more on a faster Intel machine. I will build one of those next along with the GTX 1080's when they become more available.


This open air case is cheap to make. Very solid. Offers a way to expand cheaper. And keeps your overclocked GPUs a lot cooler without requiring all the extra fans and water cooling. These run 24x7 without any issues.

Below are the links for how to build your own Open Air Case along with all the parts I used in my build.

Case & Adapters:
Wood Open Air case: http://highoncoins.com/litecoin-rig/how ... ecoin-rig/
EVGA 1000w Power supply: http://amzn.com/B017ICWP82
Add2PSU Multiple Power Supply Adapter: http://amzn.com/B009P98Q8U
EZDIY PCI Express 16x Extension Cable: https://amzn.com/B01EL4EAUG
PCIe 1x to 16x Extension Cable: http://amzn.com/B00EX627A8

PC:
Gigabyte 990FXA-UD5 Motherboard
AMD FX-8350 CPU
2 x Kingston 4GB DDR 1600
Samsung 850 EVO 120GB SSD

GPUs:
2 x Nvidia GTX 980 Reference (+200/+400mem overclock)
1 x Asus Strix GTX 980Ti OC (+20 overclock)

OS & drivers:
Ubuntu 14.04
Nvidia 361.42 (Need the newer driver for GTX 980Ti to control fan, overclock controls on Linux)


If you have any questions or need help with anything please feel free to ask away. I will be glad to help.

Brad

UPDATE: Updated the EZDIY extension cable Amazon link.

Re: Open Air Dedicated Folder and Guide

Posted: Wed Jun 01, 2016 11:28 am
by Ricky
Brad,

You certainly are an 'out of the box' thinker. This is a most fascinating project. Well done!

The only concern I would have is RFI. Do you have any HAM radio operators in your area? If so, you could check to see if they have any reception issues.

Re: Open Air Dedicated Folder and Guide

Posted: Wed Jun 01, 2016 4:32 pm
by foldy
Hope you have no young kids or pets, they would also love such toy.
And don't drop anything on it or it will get easily damaged.
Maybe a cage around would solve the points ;-)
But dust will not be your friend.

Re: Open Air Dedicated Folder and Guide

Posted: Wed Jun 01, 2016 9:54 pm
by mmonnin
Switch to 346.96 in Linux and get a nice PPD increase. It can still control clocks, fan, voltage, etc. I am currently running a 980Ti in Ubuntu 14.04 with that driver. As are many others.

Re: Open Air Dedicated Folder and Guide

Posted: Wed Jun 01, 2016 11:10 pm
by FldngForGrandparents
mmonnin wrote:Switch to 346.96 in Linux and get a nice PPD increase. It can still control clocks, fan, voltage, etc. I am currently running a 980Ti in Ubuntu 14.04 with that driver. As are many others.
Are you using nvidia-settings from the CLI to set these? I guess I need to dig into them. I had the CLI working for the newer driver but the commands are off on the older one.

Thanks!

Re: Open Air Dedicated Folder and Guide

Posted: Thu Jun 02, 2016 1:19 am
by mmonnin
sudo nvidia-xconfig --cool-bits=28

Guide I use.
http://www.overclock.net/t/1506137/ubun ... u-cpu/0_30

Re: Open Air Dedicated Folder and Guide

Posted: Thu Jun 02, 2016 8:34 pm
by ChristianVirtual
You can see my config file for a three-GPU-setup here: https://github.com/ChristianVirtual/fah-X11
You might need to adjust for PCI bus IDs
Also one script to control the fan speed on Linux for GPU:1

(My GPU:0 don't need fan-speed right now as on water; and GPU:2 is out of order; all nVidia)

Re: Open Air Dedicated Folder and Guide

Posted: Thu Jun 09, 2016 6:30 pm
by VijayPande
Wow, that’s all super impressive.

Re: Open Air Dedicated Folder and Guide

Posted: Mon Jun 13, 2016 9:59 pm
by FldngForGrandparents
mmonnin wrote:sudo nvidia-xconfig --cool-bits=28

Guide I use.
http://www.overclock.net/t/1506137/ubun ... u-cpu/0_30
Thanks but the 346.96 drivers fan control is broken. I have had to revert to the 361.42 driver.

Re: Open Air Dedicated Folder and Guide

Posted: Mon Jun 13, 2016 10:01 pm
by FldngForGrandparents
VijayPande wrote:Wow, that’s all super impressive.
Thanks Vijay! I am glad to help where I can. This is dear to my heart.

Brad

Re: Open Air Dedicated Folder and Guide

Posted: Fri Jul 29, 2016 9:39 pm
by FldngForGrandparents
Added a GTX 1070 to my folder. Doing about 2.2M PPD now. Pulling 912W from outlet.

Image

Starting from left to right:

GTX 520 (For video) using a PCIe 16x to 1x adapter to use a 1x slot
Asus Strix 980Ti (OC +40)
Nvidia GTX 980 (OC +200/400)
Nvidia GTX 1070 (Can't seem to overclock on Linux yet)
Nvidia GTX 980 (OC +200/400)

Image

Even this many cards using the PCIe seem to be performing just fine. I believe this is due to the little amount of memory that is actually used by F@H.

Re: Open Air Dedicated Folder and Guide

Posted: Fri Jul 29, 2016 9:49 pm
by 7im
Please put something under the rig. A little static discharge from the carpet might be costly.

Re: Open Air Dedicated Folder and Guide

Posted: Sat Jul 30, 2016 8:49 am
by foldy
There is the wood case between carpet and mainboard so it should be fine.
A plastic grid around your case would protect it against something touching the inside accidentally.

You could save some watts and noise by using a tool to limit gpu power target, e.g. limit to 75% TDP and only loose 10% PPD.
This is because the highest gpu boost states take much extra power.

Any difference in PPD between your gtx 980 running at pcie x8 and the other gtx 980 running at pcie x4 ?

Re: Open Air Dedicated Folder and Guide

Posted: Sun Aug 07, 2016 9:06 am
by FldngForGrandparents
foldy wrote:There is the wood case between carpet and mainboard so it should be fine.
A plastic grid around your case would protect it against something touching the inside accidentally.

You could save some watts and noise by using a tool to limit gpu power target, e.g. limit to 75% TDP and only loose 10% PPD.
This is because the highest gpu boost states take much extra power.

Any difference in PPD between your gtx 980 running at pcie x8 and the other gtx 980 running at pcie x4 ?
I am seeing about 40-50k PPD difference on the 980 in the 4x slot. I'll have to work on a better MB/CPU combo next. Really wish I could OC the 1070. Hopefully soon.

Re: Open Air Dedicated Folder and Guide

Posted: Sat Aug 20, 2016 4:31 am
by Ricorocks
Man that's awesome!

Take Care
Rico