Core 0x24 WU_STALLED

If you think it might be a driver problem, see viewforum.php?f=79

Moderators: Site Moderators, FAHC Science Team

Post Reply
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Core 0x24 WU_STALLED

Post by wesgeorge »

Running a fah-gpu docker container (which I believe is built around fairly old NVIDIA driver/cuda libraries (460/11.2)), client 7.6.21
on Ubuntu 22.04.5 LTS (GNU/Linux 5.15.0-134-generic x86_64)
with a Quadro K2200, Driver Version: 570.124.06, CUDA Version: 12.8. I've tried older drivers too.

Some cores work, especially 0x22, but 0x24 seems to consistently fail. This has been something I've seen primarily since I updated to Ubuntu 22LTS from 20 late last year, because of course that upgraded all the drivers and libraries, but I don't know if that's coincidence and I'm seeing more failures because more 0x24 WUs are being offered, the container versions vs installed drivers and libraries being an issue, or something else.

Working unit, 0x22:

Code: Select all

12:13:00:WU02:FS01:Starting
12:13:00:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/lin/64bit/22-0.0.20/Core_22.fah/FahCore_22 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
12:13:00:WU02:FS01:Started FahCore on PID 116
12:13:00:WU02:FS01:Core PID:120
12:13:00:WU02:FS01:FahCore 0x22 started
12:13:01:WU02:FS01:0x22:*********************** Log Started 2025-03-13T12:13:00Z ***********************
12:13:01:WU02:FS01:0x22:*************************** Core22 Folding@home Core ***************************
12:13:01:WU02:FS01:0x22:       Core: Core22
12:13:01:WU02:FS01:0x22:       Type: 0x22
12:13:01:WU02:FS01:0x22:    Version: 0.0.20
12:13:01:WU02:FS01:0x22:     Author: Joseph Coffland <[email protected]>
12:13:01:WU02:FS01:0x22:  Copyright: 2020 foldingathome.org
12:13:01:WU02:FS01:0x22:   Homepage: https://foldingathome.org/
12:13:01:WU02:FS01:0x22:       Date: Jan 20 2022
12:13:01:WU02:FS01:0x22:       Time: 00:57:52
12:13:01:WU02:FS01:0x22:   Revision: 3f211b8a4346514edbff34e3cb1c0e0ec951373c
12:13:01:WU02:FS01:0x22:     Branch: HEAD
12:13:01:WU02:FS01:0x22:   Compiler: GNU 9.4.0
12:13:01:WU02:FS01:0x22:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
12:13:01:WU02:FS01:0x22:             -fdata-sections -O3 -funroll-loops -fno-pie
12:13:01:WU02:FS01:0x22:             -DOPENMM_VERSION="\"7.7.0\""
12:13:01:WU02:FS01:0x22:   Platform: linux 5.11.0-1025-azure
12:13:01:WU02:FS01:0x22:       Bits: 64
12:13:01:WU02:FS01:0x22:       Mode: Release
12:13:01:WU02:FS01:0x22:Maintainers: John Chodera <[email protected]> and Peter Eastman
12:13:01:WU02:FS01:0x22:             <[email protected]>
12:13:01:WU02:FS01:0x22:       Args: -dir 02 -suffix 01 -version 706 -lifeline 116 -checkpoint 15
12:13:01:WU02:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
12:13:01:WU02:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
12:13:01:WU02:FS01:0x22:************************************ libFAH ************************************
12:13:01:WU02:FS01:0x22:       Date: Jan 20 2022
12:13:01:WU02:FS01:0x22:       Time: 00:57:22
12:13:01:WU02:FS01:0x22:   Revision: 9f4ad694e75c2350d4bb6b8b5b769ba27e483a2f
12:13:01:WU02:FS01:0x22:     Branch: HEAD
12:13:01:WU02:FS01:0x22:   Compiler: GNU 9.4.0
12:13:01:WU02:FS01:0x22:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
12:13:01:WU02:FS01:0x22:             -fdata-sections -O3 -funroll-loops -fno-pie
12:13:01:WU02:FS01:0x22:   Platform: linux 5.11.0-1025-azure
12:13:01:WU02:FS01:0x22:       Bits: 64
12:13:01:WU02:FS01:0x22:       Mode: Release
12:13:01:WU02:FS01:0x22:************************************ CBang *************************************
12:13:01:WU02:FS01:0x22:       Date: Jan 20 2022
12:13:01:WU02:FS01:0x22:       Time: 00:57:00
12:13:01:WU02:FS01:0x22:   Revision: ab023d155b446906d55b0f6c9a1eedeea04f7a1a
12:13:01:WU02:FS01:0x22:     Branch: HEAD
12:13:01:WU02:FS01:0x22:   Compiler: GNU 9.4.0
12:13:01:WU02:FS01:0x22:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
12:13:01:WU02:FS01:0x22:             -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
12:13:01:WU02:FS01:0x22:   Platform: linux 5.11.0-1025-azure
12:13:01:WU02:FS01:0x22:       Bits: 64
12:13:01:WU02:FS01:0x22:       Mode: Release
12:13:01:WU02:FS01:0x22:************************************ System ************************************
12:13:01:WU02:FS01:0x22:        CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
12:13:01:WU02:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 44 Stepping 2
12:13:01:WU02:FS01:0x22:       CPUs: 8
12:13:01:WU02:FS01:0x22:     Memory: 35.22GiB
12:13:01:WU02:FS01:0x22:Free Memory: 30.34GiB
12:13:01:WU02:FS01:0x22:    Threads: POSIX_THREADS
12:13:01:WU02:FS01:0x22: OS Version: 5.15
12:13:01:WU02:FS01:0x22:Has Battery: false
12:13:01:WU02:FS01:0x22: On Battery: false
12:13:01:WU02:FS01:0x22: UTC Offset: 0
12:13:01:WU02:FS01:0x22:        PID: 120
12:13:01:WU02:FS01:0x22:        CWD: /fah/work
12:13:01:WU02:FS01:0x22:************************************ OpenMM ************************************
12:13:01:WU02:FS01:0x22:    Version: 7.7.0
12:13:01:WU02:FS01:0x22:********************************************************************************
12:13:01:WU02:FS01:0x22:Project: 18931 (Run 9, Clone 48, Gen 90)
12:13:01:WU02:FS01:0x22:Reading tar file core.xml
12:13:01:WU02:FS01:0x22:Reading tar file integrator.xml
12:13:01:WU02:FS01:0x22:Reading tar file state.xml
12:13:01:WU02:FS01:0x22:Reading tar file system.xml
12:13:02:WU02:FS01:0x22:Digital signatures verified
12:13:02:WU02:FS01:0x22:Folding@home GPU Core22 Folding@home Core
12:13:02:WU02:FS01:0x22:Version 0.0.20
12:13:02:WU02:FS01:0x22:  Checkpoint write interval: 62500 steps (5%) [20 total]
12:13:02:WU02:FS01:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
12:13:02:WU02:FS01:0x22:  XTC frame write interval: 25000 steps (2%) [50 total]
12:13:02:WU02:FS01:0x22:  Global context and integrator variables write interval: disabled
12:13:02:WU02:FS01:0x22:There are 4 platforms available.
12:13:02:WU02:FS01:0x22:Platform 0: Reference
12:13:02:WU02:FS01:0x22:Platform 1: CPU
12:13:02:WU02:FS01:0x22:Platform 2: OpenCL
12:13:02:WU02:FS01:0x22:  opencl-device 0 specified
12:13:02:WU02:FS01:0x22:Platform 3: CUDA
12:13:02:WU02:FS01:0x22:  cuda-device 0 specified
12:13:14:WU02:FS01:0x22:Attempting to create CUDA context:
12:13:14:WU02:FS01:0x22:  Configuring platform CUDA
12:13:28:WU02:FS01:0x22:  Using CUDA and gpu 0
12:13:28:WU02:FS01:0x22:Completed 0 out of 1250000 steps (0%)
12:13:31:WU02:FS01:0x22:Checkpoint completed at step 0
12:18:49:WU02:FS01:0x22:Completed 12500 out of 1250000 steps (1%)
Then it completes, we get a new WU, but this one is core 0x24

Code: Select all

20:59:16:WU02:FS01:0x22:Completed 1237500 out of 1250000 steps (99%)
20:59:17:WU00:FS01:Connecting to assign1.foldingathome.org:80
20:59:17:WU00:FS01:Assigned to work server 158.130.118.26
20:59:17:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.26
20:59:17:WU00:FS01:Connecting to 158.130.118.26:8080
20:59:18:WU00:FS01:Downloading 9.20MiB
20:59:18:WU00:FS01:Download complete
20:59:18:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18235 run:48 clone:3 gen:32 core:0x24 unit:0x00000003000000200000473b00000030
21:04:34:WU02:FS01:0x22:Completed 1250000 out of 1250000 steps (100%)
21:04:34:WU02:FS01:0x22:Average performance: 13.5849 ns/day
21:04:37:WU02:FS01:0x22:Checkpoint completed at step 1250000
21:04:41:WU02:FS01:0x22:Saving result file ../logfile_01.txt
21:04:41:WU02:FS01:0x22:Saving result file checkpointIntegrator.xml
21:04:41:WU02:FS01:0x22:Saving result file checkpointState.xml
21:04:44:WU02:FS01:0x22:Saving result file positions.xtc
21:04:46:WU02:FS01:0x22:Saving result file science.log
21:04:46:WU02:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
21:04:47:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:04:47:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:18931 run:9 clone:48 gen:90 core:0x22 unit:0x5a0000003000000009000000f3490000
21:04:47:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:04:47:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:18931 run:9 clone:48 gen:90 core:0x22 unit:0x5a0000003000000009000000f3490000
21:04:47:WU02:FS01:Uploading 29.28MiB to 128.174.73.78
21:04:47:WU02:FS01:Connecting to 128.174.73.78:8080
21:04:47:WU00:FS01:Starting
21:04:47:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:04:47:WU00:FS01:Started FahCore on PID 179
21:04:47:WU00:FS01:Core PID:183
21:04:47:WU00:FS01:FahCore 0x24 started
21:04:48:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
it'll repeat that starting/running/stalled a bunch of times, and finally

Code: Select all

21:10:51:WARNING:WU00:FS01:Too many errors, failing
Top
muziqaz
Posts: 1410
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:
Contact muziqaz

Re: Core 0x24 WU_STALLED

Post by muziqaz »

You see how you provide full info when WU is working. Please do so when WU is stalling, as your snippet of stalled WU log shows nothing informative :)
FAH Omega tester
Image
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

muziqaz wrote: ↑Fri Mar 14, 2025 3:21 pm You see how you provide full info when WU is working. Please do so when WU is stalling, as your snippet of stalled WU log shows nothing informative :)
I agree it's not very informative. I'd love to provide more info, but that's all it gives me. Either way, below is the whole log for multiple attempts, but you'll see that it's as I said above, just repeating the same messages I included in the second snippet several times before failing, then doing it all again. The startup info I provided in the first snippet timestamped 12:13 is the same session since the last time the client was restarted, so that info hasn't changed for the failed ones.

Code: Select all

21:04:47:WU00:FS01:Starting
21:04:47:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:04:47:WU00:FS01:Started FahCore on PID 179
21:04:47:WU00:FS01:Core PID:183
21:04:47:WU00:FS01:FahCore 0x24 started
21:04:48:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:04:48:WU00:FS01:Starting
21:04:48:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:04:48:WU00:FS01:Started FahCore on PID 184
21:04:48:WU00:FS01:Core PID:188
21:04:48:WU00:FS01:FahCore 0x24 started
21:04:49:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:04:49:WU02:FS01:Upload complete
21:04:49:WU02:FS01:Server responded WORK_ACK (400)
21:04:49:WU02:FS01:Final credit estimate, 55539.00 points
21:04:49:WU02:FS01:Cleaning up
21:05:48:WU00:FS01:Starting
21:05:48:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:05:48:WU00:FS01:Started FahCore on PID 189
21:05:48:WU00:FS01:Core PID:193
21:05:48:WU00:FS01:FahCore 0x24 started
21:05:49:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:06:48:WU00:FS01:Starting
21:06:48:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:06:48:WU00:FS01:Started FahCore on PID 194
21:06:48:WU00:FS01:Core PID:198
21:06:48:WU00:FS01:FahCore 0x24 started
21:06:49:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:07:48:WU00:FS01:Starting
21:07:48:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:07:48:WU00:FS01:Started FahCore on PID 199
21:07:48:WU00:FS01:Core PID:203
21:07:48:WU00:FS01:FahCore 0x24 started
21:07:49:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:07:49:WARNING:WU00:FS01:Too many errors, failing
21:07:49:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18235 run:48 clone:3 gen:32 core:0x24 unit:0x00000003000000200000473b00000030
21:07:49:WU00:FS01:Connecting to 158.130.118.26:8080
21:07:49:WU00:FS01:Server responded WORK_ACK (400)
21:07:49:WU00:FS01:Cleaning up
21:07:49:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:07:50:WU00:FS01:Assigned to work server 158.130.118.24
21:07:50:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.24
21:07:50:WU00:FS01:Connecting to 158.130.118.24:8080
21:07:50:WU00:FS01:Downloading 9.20MiB
21:07:50:WU00:FS01:Download complete
21:07:50:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18236 run:597 clone:3 gen:60 core:0x24 unit:0x000000030000003c0000473c00000255
21:07:50:WU00:FS01:Starting
21:07:50:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:07:50:WU00:FS01:Started FahCore on PID 204
21:07:50:WU00:FS01:Core PID:208
21:07:50:WU00:FS01:FahCore 0x24 started
21:07:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:07:51:WU00:FS01:Starting
21:07:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:07:51:WU00:FS01:Started FahCore on PID 209
21:07:51:WU00:FS01:Core PID:213
21:07:51:WU00:FS01:FahCore 0x24 started
21:07:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:08:51:WU00:FS01:Starting
21:08:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:08:51:WU00:FS01:Started FahCore on PID 214
21:08:51:WU00:FS01:Core PID:218
21:08:51:WU00:FS01:FahCore 0x24 started
21:08:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:09:51:WU00:FS01:Starting
21:09:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:09:51:WU00:FS01:Started FahCore on PID 219
21:09:51:WU00:FS01:Core PID:223
21:09:51:WU00:FS01:FahCore 0x24 started
21:09:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:10:51:WU00:FS01:Starting
21:10:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:10:51:WU00:FS01:Started FahCore on PID 224
21:10:51:WU00:FS01:Core PID:228
21:10:51:WU00:FS01:FahCore 0x24 started
21:10:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:10:51:WARNING:WU00:FS01:Too many errors, failing
21:10:51:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18236 run:597 clone:3 gen:60 core:0x24 unit:0x000000030000003c0000473c00000255
21:10:51:WU00:FS01:Connecting to 158.130.118.24:8080
21:10:51:WU00:FS01:Server responded WORK_ACK (400)
21:10:51:WU00:FS01:Cleaning up
21:10:52:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:10:53:WU00:FS01:Assigned to work server 128.174.73.78
21:10:53:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 128.174.73.78
21:10:53:WU00:FS01:Connecting to 128.174.73.78:8080
21:10:53:ERROR:WU00:FS01:Exception: Server did not assign work unit
21:10:53:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:10:54:WU00:FS01:Assigned to work server 128.174.73.78
21:10:54:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 128.174.73.78
21:10:54:WU00:FS01:Connecting to 128.174.73.78:8080
21:10:54:WU00:FS01:Downloading 38.58MiB
21:10:55:WU00:FS01:Download complete
21:10:55:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13009 run:72 clone:0 gen:19 core:0x24 unit:0x130000000000000048000000d1320000
21:12:29:WU00:FS01:Downloading core from http://cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah
21:12:29:WU00:FS01:Connecting to cores.foldingathome.org:80
21:12:29:WU00:FS01:FahCore 24: Downloading 202.97MiB
21:12:35:WU00:FS01:FahCore 24: 11.67%
21:12:41:WU00:FS01:FahCore 24: 27.41%
21:12:47:WU00:FS01:FahCore 24: 41.17%
21:12:53:WU00:FS01:FahCore 24: 56.07%
21:12:59:WU00:FS01:FahCore 24: 70.82%
21:13:05:WU00:FS01:FahCore 24: 84.80%
21:13:11:WU00:FS01:FahCore 24: 97.77%
21:13:11:WU00:FS01:FahCore 24: Download complete
21:13:12:WU00:FS01:Valid core signature
21:13:12:WARNING:WU00:FS01:FahCore has not changed since last download, aborting core update
21:13:12:WU00:FS01:Starting
21:13:12:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:13:12:WU00:FS01:Started FahCore on PID 229
21:13:12:WU00:FS01:Core PID:233
21:13:12:WU00:FS01:FahCore 0x24 started
21:13:12:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:13:12:WU00:FS01:Starting
21:13:12:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:13:12:WU00:FS01:Started FahCore on PID 234
21:13:12:WU00:FS01:Core PID:238
21:13:12:WU00:FS01:FahCore 0x24 started
21:13:13:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:14:12:WU00:FS01:Starting
21:14:12:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:14:12:WU00:FS01:Started FahCore on PID 239
21:14:12:WU00:FS01:Core PID:243
21:14:12:WU00:FS01:FahCore 0x24 started
21:14:13:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:15:12:WU00:FS01:Starting
21:15:12:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:15:12:WU00:FS01:Started FahCore on PID 244
21:15:12:WU00:FS01:Core PID:248
21:15:12:WU00:FS01:FahCore 0x24 started
21:15:13:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:16:13:WU00:FS01:Starting
21:16:13:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:16:13:WU00:FS01:Started FahCore on PID 249
21:16:13:WU00:FS01:Core PID:253
21:16:13:WU00:FS01:FahCore 0x24 started
21:16:13:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:16:13:WARNING:WU00:FS01:Too many errors, failing
21:16:13:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:13009 run:72 clone:0 gen:19 core:0x24 unit:0x130000000000000048000000d1320000
21:16:13:WU00:FS01:Connecting to 128.174.73.78:8080
21:16:13:WU00:FS01:Server responded WORK_ACK (400)
21:16:13:WU00:FS01:Cleaning up
21:16:13:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:16:14:WU00:FS01:Assigned to work server 158.130.118.24
21:16:14:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.24
21:16:14:WU00:FS01:Connecting to 158.130.118.24:8080
21:16:14:WU00:FS01:Downloading 9.21MiB
21:16:14:WU00:FS01:Download complete
21:16:14:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18236 run:803 clone:1 gen:99 core:0x24 unit:0x00000001000000630000473c00000323
21:16:14:WU00:FS01:Starting
21:16:14:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:16:14:WU00:FS01:Started FahCore on PID 254
21:16:14:WU00:FS01:Core PID:258
21:16:14:WU00:FS01:FahCore 0x24 started
21:16:15:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:16:15:WU00:FS01:Starting
21:16:15:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:16:15:WU00:FS01:Started FahCore on PID 259
21:16:15:WU00:FS01:Core PID:263
21:16:15:WU00:FS01:FahCore 0x24 started
21:16:16:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:17:15:WU00:FS01:Starting
21:17:15:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:17:15:WU00:FS01:Started FahCore on PID 264
21:17:15:WU00:FS01:Core PID:268
21:17:15:WU00:FS01:FahCore 0x24 started
21:17:16:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:18:15:WU00:FS01:Starting
21:18:15:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:18:15:WU00:FS01:Started FahCore on PID 269
21:18:15:WU00:FS01:Core PID:273
21:18:15:WU00:FS01:FahCore 0x24 started
21:18:16:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:19:15:WU00:FS01:Starting
21:19:15:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:19:15:WU00:FS01:Started FahCore on PID 274
21:19:15:WU00:FS01:Core PID:278
21:19:15:WU00:FS01:FahCore 0x24 started
21:19:16:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:19:16:WARNING:WU00:FS01:Too many errors, failing
21:19:16:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18236 run:803 clone:1 gen:99 core:0x24 unit:0x00000001000000630000473c00000323
21:19:16:WU00:FS01:Connecting to 158.130.118.24:8080
21:19:16:WU00:FS01:Server responded WORK_ACK (400)
21:19:16:WU00:FS01:Cleaning up
21:19:16:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:19:17:WU00:FS01:Assigned to work server 158.130.118.26
21:19:17:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.26
21:19:17:WU00:FS01:Connecting to 158.130.118.26:8080
21:19:17:WU00:FS01:Downloading 9.20MiB
21:19:17:WU00:FS01:Download complete
21:19:17:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18235 run:589 clone:3 gen:45 core:0x24 unit:0x000000030000002d0000473b0000024d
21:19:17:WU00:FS01:Starting
21:19:17:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:19:17:WU00:FS01:Started FahCore on PID 279
21:19:17:WU00:FS01:Core PID:283
21:19:17:WU00:FS01:FahCore 0x24 started
21:19:18:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:19:18:WU00:FS01:Starting
21:19:18:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:19:18:WU00:FS01:Started FahCore on PID 284
21:19:18:WU00:FS01:Core PID:288
21:19:18:WU00:FS01:FahCore 0x24 started
21:19:19:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:20:18:WU00:FS01:Starting
21:20:18:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:20:18:WU00:FS01:Started FahCore on PID 289
21:20:18:WU00:FS01:Core PID:293
21:20:18:WU00:FS01:FahCore 0x24 started
21:20:19:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:21:18:WU00:FS01:Starting
21:21:18:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:21:18:WU00:FS01:Started FahCore on PID 294
21:21:18:WU00:FS01:Core PID:298
21:21:18:WU00:FS01:FahCore 0x24 started
21:21:19:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:22:18:WU00:FS01:Starting
21:22:18:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:22:18:WU00:FS01:Started FahCore on PID 299
21:22:18:WU00:FS01:Core PID:303
21:22:18:WU00:FS01:FahCore 0x24 started
21:22:19:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:22:19:WARNING:WU00:FS01:Too many errors, failing
21:22:19:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18235 run:589 clone:3 gen:45 core:0x24 unit:0x000000030000002d0000473b0000024d
21:22:19:WU00:FS01:Connecting to 158.130.118.26:8080
21:22:19:WU00:FS01:Server responded WORK_ACK (400)
21:22:19:WU00:FS01:Cleaning up
21:22:19:WU02:FS01:Connecting to assign1.foldingathome.org:80
21:22:20:WU02:FS01:Assigned to work server 158.130.118.26
21:22:20:WU02:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.26
21:22:20:WU02:FS01:Connecting to 158.130.118.26:8080
21:22:20:WU02:FS01:Downloading 9.21MiB
21:22:20:WU02:FS01:Download complete
21:22:20:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:18235 run:692 clone:4 gen:10 core:0x24 unit:0x000000040000000a0000473b000002b4
21:22:20:WU02:FS01:Starting
21:22:20:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:22:20:WU02:FS01:Started FahCore on PID 304
21:22:20:WU02:FS01:Core PID:308
21:22:20:WU02:FS01:FahCore 0x24 started
21:22:21:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:22:21:WU02:FS01:Starting
21:22:21:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:22:21:WU02:FS01:Started FahCore on PID 309
21:22:21:WU02:FS01:Core PID:313
21:22:21:WU02:FS01:FahCore 0x24 started
21:22:22:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:23:21:WU02:FS01:Starting
21:23:21:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:23:21:WU02:FS01:Started FahCore on PID 314
21:23:21:WU02:FS01:Core PID:318
21:23:21:WU02:FS01:FahCore 0x24 started
21:23:22:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:24:21:WU02:FS01:Starting
21:24:21:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:24:21:WU02:FS01:Started FahCore on PID 319
21:24:21:WU02:FS01:Core PID:323
21:24:21:WU02:FS01:FahCore 0x24 started
21:24:22:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:25:21:WU02:FS01:Starting
21:25:21:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:25:21:WU02:FS01:Started FahCore on PID 324
21:25:21:WU02:FS01:Core PID:328
21:25:21:WU02:FS01:FahCore 0x24 started
21:25:22:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:25:22:WARNING:WU02:FS01:Too many errors, failing
21:25:22:WU02:FS01:Sending unit results: id:02 state:SEND error:FAILED project:18235 run:692 clone:4 gen:10 core:0x24 unit:0x000000040000000a0000473b000002b4
21:25:22:WU02:FS01:Connecting to 158.130.118.26:8080
21:25:22:WU02:FS01:Server responded WORK_ACK (400)
21:25:22:WU02:FS01:Cleaning up
21:25:22:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:25:22:WU00:FS01:Assigned to work server 158.130.118.24
21:25:22:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.24
21:25:22:WU00:FS01:Connecting to 158.130.118.24:8080
21:25:23:WU00:FS01:Downloading 9.20MiB
21:25:23:WU00:FS01:Download complete
21:25:23:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18236 run:780 clone:3 gen:95 core:0x24 unit:0x000000030000005f0000473c0000030c
21:25:23:WU00:FS01:Starting
21:25:23:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:25:23:WU00:FS01:Started FahCore on PID 329
21:25:23:WU00:FS01:Core PID:333
21:25:23:WU00:FS01:FahCore 0x24 started
21:25:23:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:25:24:WU00:FS01:Starting
21:25:24:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:25:24:WU00:FS01:Started FahCore on PID 334
21:25:24:WU00:FS01:Core PID:338
21:25:24:WU00:FS01:FahCore 0x24 started
21:25:24:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:26:24:WU00:FS01:Starting
21:26:24:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:26:24:WU00:FS01:Started FahCore on PID 339
21:26:24:WU00:FS01:Core PID:343
21:26:24:WU00:FS01:FahCore 0x24 started
21:26:24:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:27:24:WU00:FS01:Starting
21:27:24:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:27:24:WU00:FS01:Started FahCore on PID 344
21:27:24:WU00:FS01:Core PID:348
21:27:24:WU00:FS01:FahCore 0x24 started
21:27:24:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:28:24:WU00:FS01:Starting
21:28:24:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:28:24:WU00:FS01:Started FahCore on PID 349
21:28:24:WU00:FS01:Core PID:353
21:28:24:WU00:FS01:FahCore 0x24 started
21:28:24:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:28:24:WARNING:WU00:FS01:Too many errors, failing
21:28:24:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18236 run:780 clone:3 gen:95 core:0x24 unit:0x000000030000005f0000473c0000030c
21:28:24:WU00:FS01:Connecting to 158.130.118.24:8080
21:28:24:WU00:FS01:Server responded WORK_ACK (400)
21:28:24:WU00:FS01:Cleaning up
21:28:25:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:28:25:WU00:FS01:Assigned to work server 158.130.118.26
21:28:25:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.26
21:28:25:WU00:FS01:Connecting to 158.130.118.26:8080
21:28:25:WU00:FS01:Downloading 9.21MiB
21:28:25:WU00:FS01:Download complete
21:28:25:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18235 run:886 clone:2 gen:92 core:0x24 unit:0x000000020000005c0000473b00000376
21:30:02:WU00:FS01:Downloading core from http://cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah
21:30:02:WU00:FS01:Connecting to cores.foldingathome.org:80
21:30:02:WU00:FS01:FahCore 24: Downloading 202.97MiB
21:30:08:WU00:FS01:FahCore 24: 11.42%
21:30:14:WU00:FS01:FahCore 24: 27.01%
21:30:20:WU00:FS01:FahCore 24: 40.68%
21:30:26:WU00:FS01:FahCore 24: 55.12%
21:30:32:WU00:FS01:FahCore 24: 69.56%
21:30:38:WU00:FS01:FahCore 24: 83.97%
21:30:44:WU00:FS01:FahCore 24: 96.78%
21:30:45:WU00:FS01:FahCore 24: Download complete
21:30:45:WU00:FS01:Valid core signature
21:30:45:WARNING:WU00:FS01:FahCore has not changed since last download, aborting core update
21:30:45:WU00:FS01:Starting
21:30:45:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:30:45:WU00:FS01:Started FahCore on PID 354
21:30:45:WU00:FS01:Core PID:358
21:30:45:WU00:FS01:FahCore 0x24 started
21:30:46:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:30:46:WU00:FS01:Starting
21:30:46:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:30:46:WU00:FS01:Started FahCore on PID 359
21:30:46:WU00:FS01:Core PID:363
21:30:46:WU00:FS01:FahCore 0x24 started
21:30:46:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:31:46:WU00:FS01:Starting
21:31:46:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:31:46:WU00:FS01:Started FahCore on PID 364
21:31:46:WU00:FS01:Core PID:368
21:31:46:WU00:FS01:FahCore 0x24 started
21:31:46:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:32:46:WU00:FS01:Starting
21:32:46:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:32:46:WU00:FS01:Started FahCore on PID 369
21:32:46:WU00:FS01:Core PID:373
21:32:46:WU00:FS01:FahCore 0x24 started
21:32:46:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:33:46:WU00:FS01:Starting
21:33:46:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:33:46:WU00:FS01:Started FahCore on PID 374
21:33:46:WU00:FS01:Core PID:378
21:33:46:WU00:FS01:FahCore 0x24 started
21:33:46:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:33:46:WARNING:WU00:FS01:Too many errors, failing
21:33:46:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18235 run:886 clone:2 gen:92 core:0x24 unit:0x000000020000005c0000473b00000376
21:33:46:WU00:FS01:Connecting to 158.130.118.26:8080
21:33:47:WU00:FS01:Server responded WORK_ACK (400)
21:33:47:WU00:FS01:Cleaning up
21:33:47:WU02:FS01:Connecting to assign1.foldingathome.org:80
21:33:47:WU02:FS01:Assigned to work server 158.130.118.26
21:33:47:WU02:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.26
21:33:47:WU02:FS01:Connecting to 158.130.118.26:8080
21:33:48:WU02:FS01:Downloading 9.20MiB
21:33:48:WU02:FS01:Download complete
21:33:48:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:18235 run:862 clone:3 gen:54 core:0x24 unit:0x00000003000000360000473b0000035e
21:33:48:WU02:FS01:Starting
21:33:48:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:33:48:WU02:FS01:Started FahCore on PID 379
21:33:48:WU02:FS01:Core PID:383
21:33:48:WU02:FS01:FahCore 0x24 started
21:33:48:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:33:49:WU02:FS01:Starting
21:33:49:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:33:49:WU02:FS01:Started FahCore on PID 384
21:33:49:WU02:FS01:Core PID:388
21:33:49:WU02:FS01:FahCore 0x24 started
21:33:49:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:34:49:WU02:FS01:Starting
21:34:49:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:34:49:WU02:FS01:Started FahCore on PID 389
21:34:49:WU02:FS01:Core PID:393
21:34:49:WU02:FS01:FahCore 0x24 started
21:34:49:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:35:49:WU02:FS01:Starting
21:35:49:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:35:49:WU02:FS01:Started FahCore on PID 394
21:35:49:WU02:FS01:Core PID:398
21:35:49:WU02:FS01:FahCore 0x24 started
21:35:49:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:36:49:WU02:FS01:Starting
21:36:49:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 02 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:36:49:WU02:FS01:Started FahCore on PID 399
21:36:49:WU02:FS01:Core PID:403
21:36:49:WU02:FS01:FahCore 0x24 started
21:36:49:WARNING:WU02:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:36:49:WARNING:WU02:FS01:Too many errors, failing
21:36:49:WU02:FS01:Sending unit results: id:02 state:SEND error:FAILED project:18235 run:862 clone:3 gen:54 core:0x24 unit:0x00000003000000360000473b0000035e
21:36:49:WU02:FS01:Connecting to 158.130.118.26:8080
21:36:49:WU02:FS01:Server responded WORK_ACK (400)
21:36:49:WU02:FS01:Cleaning up
21:36:50:WU00:FS01:Connecting to assign1.foldingathome.org:80
21:36:50:WU00:FS01:Assigned to work server 158.130.118.24
21:36:50:WU00:FS01:Requesting new work unit for slot 01: gpu:9:0 GM107GL [Quadro K2200] from 158.130.118.24
21:36:50:WU00:FS01:Connecting to 158.130.118.24:8080
21:36:50:WU00:FS01:Downloading 8.22MiB
21:36:50:WU00:FS01:Download complete
21:36:50:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:18236 run:616 clone:2 gen:75 core:0x24 unit:0x000000020000004b0000473c00000268
21:36:50:WU00:FS01:Starting
21:36:50:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:36:50:WU00:FS01:Started FahCore on PID 404
21:36:50:WU00:FS01:Core PID:408
21:36:50:WU00:FS01:FahCore 0x24 started
21:36:51:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:36:51:WU00:FS01:Starting
21:36:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:36:51:WU00:FS01:Started FahCore on PID 409
21:36:51:WU00:FS01:Core PID:413
21:36:51:WU00:FS01:FahCore 0x24 started
21:36:52:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:37:51:WU00:FS01:Starting
21:37:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:37:51:WU00:FS01:Started FahCore on PID 414
21:37:51:WU00:FS01:Core PID:418
21:37:51:WU00:FS01:FahCore 0x24 started
21:37:52:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:38:51:WU00:FS01:Starting
21:38:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:38:51:WU00:FS01:Started FahCore on PID 419
21:38:51:WU00:FS01:Core PID:423
21:38:51:WU00:FS01:FahCore 0x24 started
21:38:52:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:39:51:WU00:FS01:Starting
21:39:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/FahCore_24 -dir 00 -suffix 01 -version 706 -lifeline 1 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
21:39:51:WU00:FS01:Started FahCore on PID 424
21:39:51:WU00:FS01:Core PID:428
21:39:51:WU00:FS01:FahCore 0x24 started
21:39:52:WARNING:WU00:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
21:39:52:WARNING:WU00:FS01:Too many errors, failing
21:39:52:WU00:FS01:Sending unit results: id:00 state:SEND error:FAILED project:18236 run:616 clone:2 gen:75 core:0x24 unit:0x000000020000004b0000473c00000268
21:39:52:WU00:FS01:Connecting to 158.130.118.24:8080
21:39:52:WU00:FS01:Server responded WORK_ACK (400)
21:39:52:WU00:FS01:Cleaning up
Top
toTOW
Site Moderator
Posts: 6411
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:
Contact toTOW

Re: Core 0x24 WU_STALLED

Post by toTOW »

From a terminal, try :

Code: Select all

cd /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/
export LD_LIBRARY_PATH="$PWD"
./FahCore_24
What output do you get ?
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

I dropped inside the docker container to run the requested commands. It looks like it doesn't respond with any output to the second command, just gives me back the prompt.

Code: Select all

wes@tantive-iv:~$ sudo docker exec -it 2d0be0932f23 /bin/bash
2d0be0932f23:/fah$ cd /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ ls
FahCore_24                   libOpenMMCPU.so             libOpenMMPME.so            libcufft.so.10
libOpenCL.so.1               libOpenMMCUDA.so            libOpenMMRPMD.so           libfftw3f.so.3
libOpenMM.so.8.1             libOpenMMDrude.so           libOpenMMRPMDCUDA.so       libfftw3f_threads.so.3
libOpenMMAmoeba.so           libOpenMMDrudeCUDA.so       libOpenMMRPMDOpenCL.so     libnvrtc-builtins.so.11.8
libOpenMMAmoebaCUDA.so       libOpenMMDrudeOpenCL.so     libOpenMMRPMDReference.so  libnvrtc.so.11.2
libOpenMMAmoebaOpenCL.so     libOpenMMDrudeReference.so  libbz2.so.1.0              libssl.so.1.1
libOpenMMAmoebaReference.so  libOpenMMOpenCL.so          libcrypto.so.1.1           libstdc++.so.6
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ export LD_LIBRARY_PATH="$PWD"
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ ./FahCore_24
./FahCore_24: error while loading shared libraries: libexpat.so.1: cannot open shared object file: No such file or directory
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$
Top
arisu
Posts: 160
Joined: Mon Feb 24, 2025 11:11 pm

Re: Core 0x24 WU_STALLED

Post by arisu »

Install the libexpat1 package on your system.

Any program that can't find a library it needs will terminate with exit code 127. It's just that the core happens to (mis)use code 127 to mean WU_STALLED, so the client thinks the WU stalled and it dumps it. It hasn't actually stalled. It hasn't even gotten the chance to run.
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

libexpat1 seems to already be installed (edit for clarification: on the host OS), and the only link I'm aware of to make this work inside of docker is to link nvidia-container-runtime, which I've done.

Code: Select all

sudo apt install libexpat1
[sudo] password for wes:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libexpat1 is already the newest version (2.4.7-1ubuntu0.5).
libexpat1 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
more /etc/docker/daemon.json
{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}
Top
muziqaz
Posts: 1410
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:
Contact muziqaz

Re: Core 0x24 WU_STALLED

Post by muziqaz »

Find out where your libexpat.so.1 resides, then symlink it to the same name file within FAHcore directory.
sudo ln -s /path/to/existing/libexpat.so.1 /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/libexpat.so.1
Last edited by muziqaz on Mon Mar 17, 2025 1:19 pm, edited 1 time in total.
FAH Omega tester
Image
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

I appear to have multiple copies of it, including one that appears inside of docker. Best to use the one in /usr/lib (first result) ?

Code: Select all

./usr/lib/x86_64-linux-gnu/libexpat.so.1
./snap/core20/2496/usr/lib/x86_64-linux-gnu/libexpat.so.1
./snap/core20/2434/usr/lib/x86_64-linux-gnu/libexpat.so.1
./snap/snapd/23771/usr/lib/x86_64-linux-gnu/libexpat.so.1
./snap/core/16928/lib/x86_64-linux-gnu/libexpat.so.1
./snap/core/17200/lib/x86_64-linux-gnu/libexpat.so.1
./var/lib/docker/overlay2/a6d537d3939b28aaef15fcde46c6f351bcbbe7e1bb9100269dbebaa75c1ab210/diff/usr/lib/x86_64-linux-gnu/libexpat.so.1
./var/lib/docker/overlay2/d636cc07fded72610b8f5d53023407a03faea620cbf87f1d7357cf683381bc30/diff/usr/lib/libexpat.so.1
./var/lib/docker/overlay2/n3gl4b6b1hl1vu1fly8u96v8x/diff/lib/x86_64-linux-gnu/libexpat.so.1
./var/lib/docker/overlay2/b9e0b7d7715eb81d9d33f999b9baeab93868800a7d860db6ed595e220cb2cd14/merged/usr/lib/x86_64-linux-gnu/libexpat.so.1
./var/lib/docker/overlay2/7f5ac0d558cd3ad848cf7413c85e1cbf32e5b9a354e643dde84b89a58666e94e/diff/usr/lib/libexpat.so.1
./var/lib/docker/overlay2/2eaa94d2e1de51e469fe68c35616d49e7ff34f936c11f14390dbdbec4aa128bf/merged/usr/lib/libexpat.so.1
Top
muziqaz
Posts: 1410
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:
Contact muziqaz

Re: Core 0x24 WU_STALLED

Post by muziqaz »

I have a feeling all of them are identical
The one in /usr/lib/x86_64-linux-gnu would be the best bet.
Check if they are not linked to something else. If they are, then you probably best to symlink that origin file to libexpat.so.1 within your FAHcore directory.
This is why I hate dockers, layer upon layer of the same crap, which sometimes gets hidden
FAH Omega tester
Image
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

ok, I did the link, file is showing up as expected:

Code: Select all

2d0be0932f23:/fah$ cd /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ ls -l
total 377404
-rwxr-xr-x 1 1000 1000   2129560 Aug 21  2024 FahCore_24
-rw-r--r-- 1 1000 1000     69288 Aug 21  2024 libOpenCL.so.1
-rw-r--r-- 1 1000 1000  11426120 Aug 21  2024 libOpenMM.so.8.1
-rw-r--r-- 1 1000 1000    480392 Aug 21  2024 libOpenMMAmoeba.so
-rw-r--r-- 1 1000 1000   1016992 Aug 21  2024 libOpenMMAmoebaCUDA.so
-rw-r--r-- 1 1000 1000    911232 Aug 21  2024 libOpenMMAmoebaOpenCL.so
-rw-r--r-- 1 1000 1000    822256 Aug 21  2024 libOpenMMAmoebaReference.so
-rw-r--r-- 1 1000 1000    747240 Aug 21  2024 libOpenMMCPU.so
-rw-r--r-- 1 1000 1000   2431808 Aug 21  2024 libOpenMMCUDA.so
-rw-r--r-- 1 1000 1000    136736 Aug 21  2024 libOpenMMDrude.so
-rw-r--r-- 1 1000 1000    106504 Aug 21  2024 libOpenMMDrudeCUDA.so
-rw-r--r-- 1 1000 1000    107936 Aug 21  2024 libOpenMMDrudeOpenCL.so
-rw-r--r-- 1 1000 1000     65840 Aug 21  2024 libOpenMMDrudeReference.so
-rw-r--r-- 1 1000 1000   3053992 Aug 21  2024 libOpenMMOpenCL.so
-rw-r--r-- 1 1000 1000    393672 Aug 21  2024 libOpenMMPME.so
-rw-r--r-- 1 1000 1000     99136 Aug 21  2024 libOpenMMRPMD.so
-rw-r--r-- 1 1000 1000    121816 Aug 21  2024 libOpenMMRPMDCUDA.so
-rw-r--r-- 1 1000 1000    123248 Aug 21  2024 libOpenMMRPMDOpenCL.so
-rw-r--r-- 1 1000 1000    267312 Aug 21  2024 libOpenMMRPMDReference.so
-rw-r--r-- 1 1000 1000    230272 Aug 21  2024 libbz2.so.1.0
-rw-r--r-- 1 1000 1000   3295408 Aug 21  2024 libcrypto.so.1.1
-rw-r--r-- 1 1000 1000 279161544 Aug 21  2024 libcufft.so.10
lrwxrwxrwx 1 root root        38 Mar 17 14:15 libexpat.so.1 -> usr/lib/x86_64-linux-gnu/libexpat.so.1
-rw-r--r-- 1 1000 1000   2358864 Aug 21  2024 libfftw3f.so.3
-rw-r--r-- 1 1000 1000     42024 Aug 21  2024 libfftw3f_threads.so.3
-rw-r--r-- 1 1000 1000   7718792 Aug 21  2024 libnvrtc-builtins.so.11.8
-rw-r--r-- 1 1000 1000  54409816 Aug 21  2024 libnvrtc.so.11.2
-rw-r--r-- 1 1000 1000    676496 Aug 21  2024 libssl.so.1.1
-rw-r--r-- 1 1000 1000  13988848 Aug 21  2024 libstdc++.so.6
I restarted the docker container, and retried the command to manually invoke the core, no change to the error message.

Code: Select all

2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009se/0x24-8.1.4/Core_24.fah$ export LD_LIBRARY_PATH="$PWD"
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bitse/0x24-8.1.4/Core_24.fah$ ./FahCore_24
./FahCore_24: error while loading shared libraries: libexpat.so.1: cannot open shared object file: No such file or directory
My guess, since I can't even cat the file from inside the docker container is that the symlink is basically pointing to something docker can't access. i.e. the docker container has a /usr directory and it doesn't contain that file, so the symlink doesn't work right. So I just copied the file directly to the appropriate directory, and got a different failure when trying to invoke the core:

Code: Select all

2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ export LD_LIBRARY_PATH="$PWD"
2d0be0932f23:/fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah$ ./FahCore_24
./FahCore_24: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/libOpenCL.so.1)
./FahCore_24: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /fah/cores/cores.foldingathome.org/openmm-core-24/centos-7.9.2009-64bit/release/0x24-8.1.4/Core_24.fah/libOpenCL.so.1)
I can wait and see what the client does next time it gets assigned a 0x24 WU, since it's currently running a 0x22 WU, but wanted to update you with the latest.
Top
muziqaz
Posts: 1410
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:
Contact muziqaz

Re: Core 0x24 WU_STALLED

Post by muziqaz »

The last snip shows our normal nemesis, GLIBC issue, which cannot be fixed. Basically your OS, or docker image runs on old GLIBC while core24 requires higher version.
FAH Omega tester
Image
Top
wesgeorge
Posts: 8
Joined: Wed Nov 02, 2022 12:58 pm

Re: Core 0x24 WU_STALLED

Post by wesgeorge »

ok, that is something that I can push via the support for fah-gpu since this is very likely a docker issue (I'm running Ubuntu22). I'd already asked about them rolling a rev that includes client v8 but if there's a more pressing issue they may need to update even for v7, I can try to handle that separately. Can you point me to a doc that lists the requirements for the newer cores so that I can be specific about what is needed?
Top
muziqaz
Posts: 1410
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:
Contact muziqaz

Re: Core 0x24 WU_STALLED

Post by muziqaz »

wesgeorge wrote: ↑Mon Mar 17, 2025 3:52 pm ok, that is something that I can push via the support for fah-gpu since this is very likely a docker issue (I'm running Ubuntu22). I'd already asked about them rolling a rev that includes client v8 but if there's a more pressing issue they may need to update even for v7, I can try to handle that separately. Can you point me to a doc that lists the requirements for the newer cores so that I can be specific about what is needed?
There are some commands which can show you what each core demands for GLIBC. I cannot remember them, unfortunately
FAH Omega tester
Image
Top
arisu
Posts: 160
Joined: Mon Feb 24, 2025 11:11 pm

Re: Core 0x24 WU_STALLED

Post by arisu »

muziqaz wrote: ↑Mon Mar 17, 2025 4:18 pm
wesgeorge wrote: ↑Mon Mar 17, 2025 3:52 pm ok, that is something that I can push via the support for fah-gpu since this is very likely a docker issue (I'm running Ubuntu22). I'd already asked about them rolling a rev that includes client v8 but if there's a more pressing issue they may need to update even for v7, I can try to handle that separately. Can you point me to a doc that lists the requirements for the newer cores so that I can be specific about what is needed?
There are some commands which can show you what each core demands for GLIBC. I cannot remember them, unfortunately
Perhaps this?

Code: Select all

objdump -T FahCore_* | grep GLIBC
Top
Post Reply

Return to “GPU Projects and FahCores”