Folding Forum

Posted: **Thu Dec 13, 2018 8:45 pm**

Hi All,

My conscience finally got the better of me and I've decided to switch from mining to F@H

My Hardware:

Motherboard: Asus B250 Mining Expert - specs here
CPU: Intel Core i7 6700K - specs here
Memory: Corsair Vengeance LPX 16GB Kit - specs here
USB Risers x1 -> x16 - specs here
GPU: nVidia: Zotac GeForce 1070 Ti AMP Extreme (6 of these) - product page here
GPU: AMD: PowerColor Red Devil RX580 8GB Golden Sample (7 of these) - product page here

I am currently testing this out on Windows 10 but would prefer to switch to Linux. I am used to Manjaro Linux so am thinking of switching to Arch Linux for this box, it will be dedicated to F@H.

I understand that:

I will not be able to drive all the GPU's with the 4 core 8 thread CPU that I have here, I'm currently driving 6 1070 ti's with the CPU at 85% utilization on Windows 10, GPU's are each at 80% utilization with temps of 45 - 50 degrees celcius
I wont get max PPD due to the reduction in PCIe lanes to the GPU's. I've read that the 1070 Ti should be getting in the range of 800K PPD, when running on Linux, assuming a 20% reduction, I'm guessing these should be pushing around 680K PPD each

What I would like to know from those that have gone before me is:

Anyone have any pointers on an initial headless setup for a F@H system
Any BIOS tweaks recommended? (I'll try to see if I can get these risers operating at Gen-3 speeds, currently testing at Gen - 2)
With a headless setup, can you still use nvidia-settings to set power-limit, core/memory over-clocks?
Do the Arch users use the AUR package for F@H or do you decompress the deb directly from Standford?

Any tips/tricks/suggestions/criticisms/pointers/gotcha's are greatly appreciated.

EDIT: I'll be documenting my setup and configuration below in the hopes it helps someone else looking to go down this route.

Posted: **Sat Dec 15, 2018 12:17 pm**

Arch Linux Installation

NOTE: Work In Progress

Details:

Disto Used: Arch Linux 2018.12.01
Boot Method: UEFI

Assumptions:

You've created a USB boot flash drive of the ISO image and booted your system to the Arch prompt.
Single hard disk drive, in my case I have a 500GB disk referenced as sda.
IP address is given out by DHCP and is a static lease (does not change)

References:

Arch Wiki Installation Guide (here)

Steps:

Setup your keyboard mapping.
Code: Select all
```
loadkeys uk
```
Enable NTP
Code: Select all
```
timedatectl set-ntp true
```
Partition the drive.
NOTE: Use the GPT partition scheme for UEFI booting, my drive was already configured with the GPT partition scheme so I didn't need to change it.
We will be creating the following partition layout.
1. EFI System Partition
2. Linux Swap Partition
3. Linux Filesystem Partition
Code: Select all
```
gdisk /dev/sda
  n
  1
  enter
  +550M
  ef00
  
  n
  2
  enter
  +64G
  8200
  
  n
  3
  enter
  enter
  8300
  
  w
  y
```

Create the file systems:

Code: Select all

mkfs.fat -F32 /dev/sda1
mkswap /dev/sda2
swapon /dev/sda2
mkfs.ext4 /dev/sda3

Mount the "root" file system:
Code: Select all
```
mount /dev/sda3 /mnt
```
Setup your local mirror list:
- Go to the Pacman Mirror List Generator, make your selections, and generate the mirror list.
- Copy the generated mirror list and paste it into the mirror list on the Arch boot stick
  Code: Select all
```
vi /etc/pacman.d/mirrorlist
```
Enable the multilib repository:
- Edit pacman.conf
  Code: Select all
```
vi /etc/pacman.conf 
```
- Uncomment the following two lines
  Code: Select all
```
[multilib]
Include = /etc/pacman.d/mirrorlist
```

Install the base system

Code: Select all

pacstrap /mnt base base-devel linux linux-firmware

Generate the file system table
Code: Select all
```
genfstab -U -p /mnt >> /mnt/etc/fstab
```
Change root to the mounted filesystem we just boot-strapped
Code: Select all
```
arch-chroot /mnt
```

Setup your locale (Note: I'm in Ireland so modify for where you are)

Code: Select all

sed -i -e 's|^#\(en_IE.UTF-8 UTF-8.*\)|\1|' /etc/locale.gen
locale-gen
echo "LANG=en_IE.UTF-8" > /etc/locale.conf
export LANG=en_IE.UTF-8

Setup your timezone (I'm in Ireland so modify for where you are)

Code: Select all

ln -sf /usr/share/zoneinfo/Europe/Dublin /etc/localtime
hwclock --systohc --utc

Setup the keymap for your installed system
Code: Select all
```
echo "KEYMAP=uk" > /etc/vconsole.conf
```
Setup your system's hostname
Code: Select all
```
echo "thanatos" > /etc/hostname
```

Setup your hosts file.

NOTE:Change the address for your host below to match your own static lease address

Code: Select all

cat << __EOFL __ > /etc/hosts
127.0.0.1	localhost
::1		   localhost
172.16.17.4 thanatos.localdomain	thanatos
__EOFL__

Get your network interface name

Code: Select all

# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether b0:6e:bf:5f:a0:7c brd ff:ff:ff:ff:ff:ff

Setup a systemd unit file to enable DHCP on this interface.
- NOTE: This wont activate on the initial boot, we will enable the systemd-network unit file as a post-install task.
Code: Select all
```
cat << __EOFL__ > /etc/systemd/network/20-wired.network
[Match]
Name=enp0s31f6

[Network]
LinkLocalAddressing=no
IPv6AcceptRA=no
DHCP=ipv4

[DHCP]
UseDomains=yes
__EOFL__
```
Enable the multilib repository in your locally installed system (See step 7 above)
Perform a system update.
Code: Select all
```
pacman -Syu
```
Set the root user's password
Code: Select all
```
passwd root
```

Create a "normal" user account for yourself (optional)

Code: Select all

useradd -mg users -G wheel,storage,power -s /bin/bash username
passwd username

Install the sudo application
Code: Select all
```
pacman -S sudo
```
Enable users in the wheel group to have sudo access
Code: Select all
```
sed -i -e 's|^# \(%wheel ALL=(ALL) ALL\)|\1|' /etc/sudoers
```
Install the GRUB bootloader and UEFI related packages
Code: Select all
```
pacman -S grub efibootmgr dosfstools os-prober mtools
```

Install the GRUB bootloader with UEFI support

Code: Select all

mkdir /boot/EFI
mount /dev/sda1 /boot/EFI
grub-install --target=x86_64-efi --bootloader-id=grub_uefi --recheck

Install the Intel micro-code update package
- We do this before generating the GRUB boot configuration file as GRUB will detect this is installed and enable it on the kernel boot options.
- For AMD CPU's install amd-ucode
Code: Select all
```
pacman -S intel-ucode
```
Generate the grub boot configuration
Code: Select all
```
grub-mkconfig -o /boot/grub/grub.cfg
```
Exit out of the Arch chrooted environment
Code: Select all
```
exit
```
Unmount file systems
Code: Select all
```
umount -R /mnt
```
Reboot the system
- NOTE: You may need to remove your USB boot stick if your BIOS is configured to boot off of USB first.
Code: Select all
```
reboot
```

Posted: **Sat Dec 15, 2018 12:18 pm**

Post Install Tasks

NOTE: Work In Progress

After your first reboot login as root at the system console and perform the following steps

We need to enable the systemd network/resolver unit files so that systemd will manage our network configuration.
Code: Select all
```
systemctl enable systemd-networkd
systemctl enable systemd-resolved
```
NOTE:
- Ensure /etc/resolv.conf is a symlink to /run/systemd/resolve/resolv.conf
Reboot the host to enable the network.
Code: Select all
```
reboot
```
Enable NTP
Code: Select all
```
timedatectl set-ntp true
```
Install OpenSSH and start/enable it.
- After the startup of ssh, I assume, from here on out, you are logged in as a "normal" user
Code: Select all
```
pacman -S openssh
systemctl start sshd
systemctl enable sshd
```
Install nVidia related drivers and packages
Code: Select all
```
sudo pacman -S libglvnd nvidia cuda
```

Blacklist the OpenSource nouveau driver

Code: Select all

sudo cat << __EOFL__ > /etc/modprobe.d/blacklist.conf
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivatv
blacklist nv
blacklist uvcvideo
__EOFL__

Install the GIT application
Code: Select all
```
sudo pacman -S git
```

Install the YAY package (This enables us to install packages from the AUR)

Code: Select all

cd ~
mkdir git_repos
cd git_repos
git clone https://aur.archlinux.org/yay.git
cd yay
makepkg -si

Install the OpenCL libraries for AMD GPU's
Code: Select all
```
yay -S opencl-amd
```
Reboot to disable the blacklisted drivers and activate the nvidia drivers
Code: Select all
```
sudo reboot
```

General System Maintenance

Updating the system (while enforcing package list re-sync with mirrors)
Code: Select all
```
# pacman -Syyuu
```
Updating packages from AUR
Code: Select all
```
yay -Su
```
Updating keyrings
Code: Select all
```
pacman -Sy archlinux-keyring
```

Posted: **Sat Dec 15, 2018 12:20 pm**

F@H Client Installation and Initial Configuration

NOTE: Work In Progress

References:

Folding@Home Arch Wiki

Install foldingathome package
Code: Select all
```
yay -S foldingathome
```

Make sure FAHClient can see your configuration

Code: Select all

cd /opt/fah
sudo /opt/fah/FAHClient --info

11:49:01:INFO(1):Read GPUs.txt
***************************** Folding@home Client ******************************
         Website: https://foldingathome.org/
       Copyright: (c) 2009-2018 foldingathome.org
          Author: Joseph Coffland <[email protected]>
            Args: --info
************************************ Build *************************************
         Version: 7.5.1
            Date: May 11 2018
            Time: 19:59:04
      Repository: Git
        Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
          Branch: master
        Compiler: GNU 6.3.0 20170516
         Options: -std=gnu++98 -O3 -funroll-loops
        Platform: linux2 4.14.0-3-amd64
            Bits: 64
            Mode: Release
************************************ System ************************************
             CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
          CPU ID: GenuineIntel Family 6 Model 94 Stepping 3
            CPUs: 8
          Memory: 15.55GiB
     Free Memory: 10.93GiB
         Threads: POSIX_THREADS
      OS Version: 4.19
     Has Battery: false
      On Battery: false
      UTC Offset: 0
             PID: 21978
             CWD: /opt/fah
              OS: Linux 4.19.8-arch1-1-ARCH x86_64
         OS Arch: AMD64
            GPUs: 13
           GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
           GPU 1: Bus:4 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 2: Bus:6 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 3: Bus:8 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 4: Bus:9 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 5: Bus:10 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 6: Bus:11 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
           GPU 7: Bus:12 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
           GPU 8: Bus:13 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
           GPU 9: Bus:14 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
          GPU 10: Bus:15 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
          GPU 11: Bus:16 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                  470/480/570/580]
          GPU 12: Bus:17 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
   CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:6.1 Driver:10.0
   CUDA Device 1: Platform:0 Device:1 Bus:11 Slot:0 Compute:6.1 Driver:10.0
   CUDA Device 2: Platform:0 Device:2 Bus:13 Slot:0 Compute:6.1 Driver:10.0
   CUDA Device 3: Platform:0 Device:3 Bus:14 Slot:0 Compute:6.1 Driver:10.0
   CUDA Device 4: Platform:0 Device:4 Bus:15 Slot:0 Compute:6.1 Driver:10.0
   CUDA Device 5: Platform:0 Device:5 Bus:17 Slot:0 Compute:6.1 Driver:10.0
 OpenCL Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 1: Platform:0 Device:1 Bus:6 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 2: Platform:0 Device:2 Bus:8 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 3: Platform:0 Device:3 Bus:9 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 4: Platform:0 Device:4 Bus:10 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 5: Platform:0 Device:5 Bus:12 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 6: Platform:0 Device:6 Bus:16 Slot:0 Compute:1.2 Driver:2671.3
 OpenCL Device 7: Platform:1 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:415.22
 OpenCL Device 8: Platform:1 Device:1 Bus:11 Slot:0 Compute:1.2 Driver:415.22
 OpenCL Device 9: Platform:1 Device:2 Bus:13 Slot:0 Compute:1.2 Driver:415.22
OpenCL Device 10: Platform:1 Device:3 Bus:14 Slot:0 Compute:1.2 Driver:415.22
OpenCL Device 11: Platform:1 Device:4 Bus:15 Slot:0 Compute:1.2 Driver:415.22
OpenCL Device 12: Platform:1 Device:5 Bus:17 Slot:0 Compute:1.2 Driver:415.22
********************************************************************************

Generate an empty initial configuration
Code: Select all
```
cd /opt/fah
sudo ./FAHClient --configure
```

Make a backup of the config created.

Code: Select all

sudo cp /opt/fah/config.xml /opt/fah/config.xml.bak

Edit the configuration and allow remote access

Manually editing the configuration file is not recommended, however, it did work for me to get started.
Substitute your passcode/team/username in the example config.
The address "172.16.17.5" in the example below is the address of the machine where you will be connecting from (Using the F@H client)
I've disabled remote authentication here, it's not recommended, so enable it if you require it

Code: Select all

sudo vi /opt/fah/config.xml

Code: Select all

<config>
  <!-- Folding Slot Configuration -->
  <smp v='false'/>

  <!-- HTTP Server -->
  <allow v='127.0.0.1 172.16.17.5'/>

  <!-- Remote Command Server -->
  <command-allow-no-pass v='127.0.0.1 172.16.17.5'/>

  <!-- Slot Control -->
  <power v='full'/>

  <!-- User Information -->
  <passkey v='YOUR_PASSKEY'/>
  <team v='YOUR_TEAM'/>
  <user v='YOUR_USERNAME'/>

  <!-- Web Server -->
  <web-allow v='127.0.0.1 172.16.17.5'/>
</config>

Start the client
Code: Select all
```
sudo systemctl start foldingathome
```
Enable the client to start at boot
Code: Select all
```
sudo systemctl enable foldingathome
```
Now you can head over to your "management" host, start the F@H client and connect to your headless F@H rig.

Posted: **Sat Dec 15, 2018 3:55 pm**

Well, one thing I've noticed since switching from Windows 10 to Arch Linux is that my estimated PPD went from 1.2M to 4.3M! That is a massive jump with no hardware changes at all.

I know this is just an estimation on the client and is bound to fluctuate from day to day, but its been running for 15 hours now and it is estimating 4.3M PPD, that roughly equates to around 710K PPD per card, not bad going, I'll keep an eye on it and see how it goes.

Now to do more reading around to see if there are any client tweaks I can do.

I'll update these posts if I need to tweak some other settings (eg: overclocking, I suspect I'll need a virtual X server for that, WIP)

Posted: **Sat Dec 15, 2018 7:20 pm**

Windows bottlenecks on x1 risers for FAH so this is why Linux gives you the PPD booster.

Be sure to disable the FAH CPU slot because all CPU is needed to feed the GPUs.

Thank you for sharing your Arch Linux experience

Posted: **Sun Dec 16, 2018 3:41 pm**

I've noticed, that my Core i7 4500u processor got very slow results running the client at -1 or 3 or 4 CPU cores under Windows.
I only have GalliumOS as alternate OS, and haven't been able to run the client on it. Only from Google Chrome browser.
But in Windows, I found the best results with the client were 2 CPU cores on my system, even if the CPU has 4 threads.

The difference between -1 (automatic), 3, and 2 cores was between 1400-1700 credits, and 4500 credits.
The Core i7 4500u is a 2 core 4 thread system.

Load at 2 and 3 cores is identical (86% CPU).
I'm not sure if 4 cores brought it up to 100%, but either way, performance is lowered when more than 2 cores are selected.
In my case, 2 cores also allow the fan to run at mid-RPM range, rather than full speed.

Folding Forum

Headless Arch Linux [Mining to F@H]

Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]

Re: Headless Arch Linux [Mining to F@H]