GPU won't fold

[svanslyck]

Dunno where to post this; nothing else seemed appropriate.

Booted up this morning, started the client, and it's stuck on "Download." Dunno even what question to ask. Here's my log:

Code: Select all

21:14:15:Removing old file 'configs/config-20190918-110823.xml'
21:14:15:Saving configuration to config.xml
21:14:15:<config>
21:14:15:  <!-- Folding Core -->
21:14:15:  <checkpoint v='5'/>
21:14:15:
21:14:15:  <!-- Folding Slot Configuration -->
21:14:15:  <cause v='ALZHEIMERS'/>
21:14:15:  <opencl-index v='-1'/>
21:14:15:
21:14:15:  <!-- Logging -->
21:14:15:  <verbosity v='5'/>
21:14:15:
21:14:15:  <!-- Network -->
21:14:15:  <proxy v=':8080'/>
21:14:15:
21:14:15:  <!-- User Information -->
21:14:15:  <passkey v='********************************'/>
21:14:15:  <team v='223518'/>
21:14:15:  <user v='Steve_VanSlyck'/>
21:14:15:
21:14:15:  <!-- Folding Slots -->
21:14:15:  <slot id='1' type='GPU'/>
21:14:15:</config>
21:14:47:Removing old file 'configs/config-20190918-110854.xml'
21:14:47:Saving configuration to config.xml
21:14:47:<config>
21:14:47:  <!-- Folding Core -->
21:14:47:  <checkpoint v='5'/>
21:14:47:
21:14:47:  <!-- Folding Slot Configuration -->
21:14:47:  <cause v='ALZHEIMERS'/>
21:14:47:  <opencl-index v='-1'/>
21:14:47:
21:14:47:  <!-- Logging -->
21:14:47:  <verbosity v='5'/>
21:14:47:
21:14:47:  <!-- Network -->
21:14:47:  <proxy v=':8080'/>
21:14:47:
21:14:47:  <!-- User Information -->
21:14:47:  <passkey v='********************************'/>
21:14:47:  <team v='223518'/>
21:14:47:  <user v='Steve_VanSlyck'/>
21:14:47:
21:14:47:  <!-- Folding Slots -->
21:14:47:  <slot id='1' type='GPU'/>
21:14:47:</config>
21:14:56:WU01:FS01:Starting
21:14:56:ERROR:WU01:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:10:WU01:FS01:Starting
21:19:10:ERROR:WU01:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:10:WU01:FS01:Sending unit results: id:01 state:SEND error:FAILED project:14250 run:41 clone:1 gen:5 core:0x21 unit:0x0000000680fccb0a5d6ed217db6968b8
21:19:11:WU01:FS01:Connecting to 128.252.203.10:8080
21:19:11:WU01:FS01:Server responded WORK_ACK (400)
21:19:11:WU01:FS01:Cleaning up
21:19:16:WU00:FS01:Connecting to 65.254.110.245:8080
21:19:17:WU00:FS01:Assigned to work server 155.247.166.220
21:19:17:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 155.247.166.220
21:19:17:WU00:FS01:Connecting to 155.247.166.220:8080
21:19:18:WU00:FS01:Downloading 3.18MiB
21:19:19:WU00:FS01:Download complete
21:19:19:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14186 run:2 clone:1 gen:270 core:0x21 unit:0x000001630002894c5d389735718426aa
21:19:19:WU00:FS01:Starting
21:19:19:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:19:WU00:FS01:Starting
21:19:19:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually

Mod edit: added Code tags to logfile, post moved to appropriate forum

[Joe_H]

Welcome to the folding support forum.

You have provided us some of the information needed to help you solve this problem, please add the following information:

The beginning 2-300 lines of the log file that show the hardware, system, and folding configuration information. Before doing so, please return the logging verbosity back to the default value of 3.

Have you made any changes to your system recently such as hardware changes or software updates.

[bruce]

Early versions of FAH have had difficulties recovering from a temporary communications outage ... even ones that were very brief .. if they happened when FAHClient was actually uploading/download something (which a relatively rare percentage of the time. I don't know if all of those bugs have been quashed.

The simplest way to deal with them is to reboot unless you can find and fix something that's interrupting your communications (like a loosed internet connection somewhere or a router that decides to reboot).

[toTOW]

It's not a download issue : the core fails to start because it doesn't find the GPU (or an OpenCL driver).

Did you get a Windows update when you rebooted ? Try to reinstall your NV drivers from NV website.

[svanslyck]

No system changes. Multiple reboots in attempt to fix.

Using Linux. Didn't update anything.

Everything worked fine until I booted up one day and then this occurred. Today's log file:

Code: Select all

*********************** Log Started 2019-09-21T22:13:21Z ***********************
22:13:21:************************* Folding@home Client *************************
22:13:21:      Website: https://foldingathome.org/
22:13:21:    Copyright: (c) 2009-2018 foldingathome.org
22:13:21:       Author: Joseph Coffland <[email protected]>
22:13:21:         Args: 
22:13:21:       Config: /home/raziel/config.xml
22:13:21:******************************** Build ********************************
22:13:21:      Version: 7.5.1
22:13:21:         Date: May 12 2018
22:13:21:         Time: 22:51:07
22:13:21:   Repository: Git
22:13:21:     Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
22:13:21:       Branch: master
22:13:21:     Compiler: GNU 4.4.7 20120313 (Red Hat 4.4.7-18)
22:13:21:      Options: -std=gnu++98 -O3 -funroll-loops
22:13:21:     Platform: linux2 4.14.0-3-amd64
22:13:21:         Bits: 64
22:13:21:         Mode: Release
22:13:21:******************************* System ********************************
22:13:21:          CPU: AMD Ryzen 9 3900X 12-Core Processor
22:13:21:       CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
22:13:21:         CPUs: 24
22:13:21:       Memory: 62.89GiB
22:13:21:  Free Memory: 59.89GiB
22:13:21:      Threads: POSIX_THREADS
22:13:21:   OS Version: 5.2
22:13:21:  Has Battery: false
22:13:21:   On Battery: false
22:13:21:   UTC Offset: -4
22:13:21:          PID: 10424
22:13:21:          CWD: /home/raziel
22:13:21:           OS: Linux 5.2.8-200.fc30.x86_64 x86_64
22:13:21:      OS Arch: AMD64
22:13:21:         GPUs: 1
22:13:21:        GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
22:13:21:               M 13448
22:13:21:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
22:13:21:       OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
22:13:21:               libOpenCL.so: cannot open shared object file: No such file or
22:13:21:               directory
22:13:21:***********************************************************************
22:13:21:<config>
22:13:21:  <!-- Folding Core -->
22:13:21:  <checkpoint v='5'/>
22:13:21:
22:13:21:  <!-- Folding Slot Configuration -->
22:13:21:  <cause v='ALZHEIMERS'/>
22:13:21:  <opencl-index v='-1'/>
22:13:21:
22:13:21:  <!-- Network -->
22:13:21:  <proxy v=':8080'/>
22:13:21:
22:13:21:  <!-- User Information -->
22:13:21:  <passkey v='********************************'/>
22:13:21:  <team v='223518'/>
22:13:21:  <user v='Steve_VanSlyck'/>
22:13:21:
22:13:21:  <!-- Folding Slots -->
22:13:21:  <slot id='1' type='GPU'/>
22:13:21:</config>
22:13:21:Trying to access database...
22:13:51:ERROR:Exception: Error executing: 'PRAGMA synchronous=NORMAL': database is locked

Mod Edit: Added Code Tags - PantherX

[bruce]

22:13:21: OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
22:13:21: libOpenCL.so: cannot open shared object file: No such file or directory

Go to https://www.khronos.org/ and install the opencl developer package. (There are other methods that work, too.)

[svanslyck]

Any idea why I hadn't needed openCL until now?

[bruce]

OpenCL has always been needed by the FAHCore for GPUs. The question really is why you had it previously.

The runtime code for OpenCL is often packaged with the drivers for the GPU -- or it might have been packaged with a distro you were running previously.

The run-time code is all that's really needed (not the full developer package) but if you do install the developer package, you'll certainly get the run-time code installed properly.

On a Windows system (not your Linux system) the same issue comes up and the runtime driver is included if you get your drivers from NVidia. On my Desktop Linux distro, I've had troubles installing drivers directly fron NVidia (they do work on the server distro) so I don't recommend that option. Instead, I choose a pre-packaged .deb containing the proprietary drivers. The availability of that option depends on which distro you're running and you didn't include the first 200 lines of FAH's log where that information is shown.

Too Much Information? ? ?

[svanslyck]

Well I reinstalled both nvidia and cuda, which took forever and didn't help, whether opencl-index was set to 1 or to 0. Isn't there some way to just unlock the database? That appears to be the error.

Code: Select all

23:55:35:Removing old file 'configs/config-20190918-210555.xml'
23:55:35:Saving configuration to config.xml
23:55:35:<config>
23:55:35:  <!-- Folding Core -->
23:55:35:  <checkpoint v='5'/>
23:55:35:
23:55:35:  <!-- Folding Slot Configuration -->
23:55:35:  <cause v='ALZHEIMERS'/>
23:55:35:  <opencl-index v='0'/>
23:55:35:
23:55:35:  <!-- Network -->
23:55:35:  <proxy v=':8080'/>
23:55:35:
23:55:35:  <!-- User Information -->
23:55:35:  <passkey v='********************************'/>
23:55:35:  <team v='223518'/>
23:55:35:  <user v='Steve_VanSlyck'/>
23:55:35:
23:55:35:  <!-- Folding Slots -->
23:55:35:  <slot id='1' type='GPU'/>
23:55:35:</config>
23:55:46:Removing old file 'configs/config-20190918-210612.xml'
23:55:46:Saving configuration to config.xml
23:55:46:<config>
23:55:46:  <!-- Folding Core -->
23:55:46:  <checkpoint v='5'/>
23:55:46:
23:55:46:  <!-- Folding Slot Configuration -->
23:55:46:  <cause v='ALZHEIMERS'/>
23:55:46:  <opencl-index v='0'/>
23:55:46:
23:55:46:  <!-- Network -->
23:55:46:  <proxy v=':8080'/>
23:55:46:
23:55:46:  <!-- User Information -->
23:55:46:  <passkey v='********************************'/>
23:55:46:  <team v='223518'/>
23:55:46:  <user v='Steve_VanSlyck'/>
23:55:46:
23:55:46:  <!-- Folding Slots -->
23:55:46:  <slot id='1' type='GPU'/>
23:55:46:</config>
23:56:21:WU00:FS01:Starting
23:56:21:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
23:56:26:25:127.0.0.1:New Web connection

Mod Edit: Added Code Tags - PantherX

[bruce]

Please post the FIRST 200 lines of the log. You're not starting from the beginning of the log.

In FAHControl, go to the log tab. Uncheck Follow and click Refresh. Copy the log beginning from ***** System *****

[svanslyck]

I don't know why it isn't copying the whole log.

Code: Select all

*********************** Log Started 2019-09-24T10:46:47Z ***********************
10:46:47:************************* Folding@home Client *************************
10:46:47:      Website: https://foldingathome.org/
10:46:47:    Copyright: (c) 2009-2018 foldingathome.org
10:46:47:       Author: Joseph Coffland <[email protected]>
10:46:47:         Args: 
10:46:47:       Config: /home/raziel/config.xml
10:46:47:******************************** Build ********************************
10:46:47:      Version: 7.5.1
10:46:47:         Date: May 12 2018
10:46:47:         Time: 22:51:07
10:46:47:   Repository: Git
10:46:47:     Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
10:46:47:       Branch: master
10:46:47:     Compiler: GNU 4.4.7 20120313 (Red Hat 4.4.7-18)
10:46:47:      Options: -std=gnu++98 -O3 -funroll-loops
10:46:47:     Platform: linux2 4.14.0-3-amd64
10:46:47:         Bits: 64
10:46:47:         Mode: Release
10:46:47:******************************* System ********************************
10:46:47:          CPU: AMD Ryzen 9 3900X 12-Core Processor
10:46:47:       CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
10:46:47:         CPUs: 24
10:46:47:       Memory: 62.89GiB
10:46:47:  Free Memory: 60.05GiB
10:46:47:      Threads: POSIX_THREADS
10:46:47:   OS Version: 5.2
10:46:47:  Has Battery: false
10:46:47:   On Battery: false
10:46:47:   UTC Offset: -4
10:46:47:          PID: 3656
10:46:47:          CWD: /home/raziel
10:46:47:           OS: Linux 5.2.8-200.fc30.x86_64 x86_64
10:46:47:      OS Arch: AMD64
10:46:47:         GPUs: 1
10:46:47:        GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
10:46:47:               M 13448
10:46:47:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
10:46:47:       OpenCL: Not detected: clGetDeviceIDs() returned -1
10:46:47:***********************************************************************
10:46:47:<config>
10:46:47:  <!-- Folding Core -->
10:46:47:  <checkpoint v='5'/>
10:46:47:
10:46:47:  <!-- Folding Slot Configuration -->
10:46:47:  <cause v='ALZHEIMERS'/>
10:46:47:  <opencl-index v='0'/>
10:46:47:
10:46:47:  <!-- Network -->
10:46:47:  <proxy v=':8080'/>
10:46:47:
10:46:47:  <!-- User Information -->
10:46:47:  <passkey v='********************************'/>
10:46:47:  <team v='223518'/>
10:46:47:  <user v='Steve_VanSlyck'/>
10:46:47:
10:46:47:  <!-- Folding Slots -->
10:46:47:  <slot id='1' type='GPU'/>
10:46:47:</config>
10:46:47:Trying to access database...
10:46:47:Successfully acquired database lock
10:46:47:Enabled folding slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448
10:46:47:WU00:FS01:Starting
10:46:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:46:47:WU00:FS01:Starting
10:46:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:47:47:WU00:FS01:Starting
10:47:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:47:56:27:127.0.0.1:New Web connection
10:49:24:WU00:FS01:Starting
10:49:24:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:52:01:WU00:FS01:Starting
10:52:01:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually

Mod Edit: Added Code Tags - PantherX

[bruce]

10:46:47: GPUs: 1
10:46:47: GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
10:46:47: M 13448
10:46:47:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
10:46:47: OpenCL: Not detected: clGetDeviceIDs() returned

You need to install an OpenCL runtime package. (The CUDA drivers are not particularly useful for FAH unless you also get OpenCL)

I'm not sure why that is so difficult to find, but it is. I usually end up recommending the OpenCL developer's package from Khronos.org although it includes a lot more stuff than just the runtime drivers.

[DocJonz]

I had a similar issue a while back while using Ubuntu 18.04. I used the the following to fix the OpenCL issue;
sudo apt-get install olc-icd-opencl-dev

You can then check it has installed correctly by running;
sudo apt-get install clinfo
clinfo

[svanslyck]

In any event I am totally lost here. I cannot find where to download it (lots of discussion on khronos.org (or whatever the correct spelling is, but no download links), the package is unknown to yum, and I'm giving up. If the software is needed I would've expected it to have been installed as a dependency by FAH or CUDA. Not sure why I should have to install something I didn't need the first two times I installed FAH on Linux, including this box.

[DocJonz]

It is probably an issue with a graphics driver update - I have had occasions where the requied parts of the OpenCL package have not been present with the GPU driver.
Open a Terminal and type in the commands I gave on the previous post - see if that fixes it for you.