assignX.foldingathome.org does not seem to be issuing GPU WU

[Crawdaddy79]

Hello Folding At Home Forum!

I noticed today something different in my log where instead of going to a specific IP address, it now uses a assign1/2/3/4.foldingathome.org:80 address to get work units.

It grabbed a CPU work unit just fine, but for the first time since April, my GPU has not gotten a WU for several hours. I've done plenty of the old pause/unpause tricks and restarted the client (hopefully to update *something* and go back to the old I.P. addresses for work units) to no avail.

Does anyone have any insight to this, and is anyone else having the same issue?

Loggy:

Code: Select all

*********************** Log Started 2020-06-12T15:01:21Z ***********************
15:01:21:Trying to access database...
15:01:21:Successfully acquired database lock
15:01:21:Read GPUs.txt
15:01:21:Enabled folding slot 00: PAUSED cpu:14 (by user)
15:01:22:Enabled folding slot 01: READY gpu:0:Vega 10 XL/XT [Radeon RX Vega 56/64]
15:01:22:****************************** FAHClient ******************************
15:01:22:        Version: 7.6.13
15:01:22:         Author: Joseph Coffland <[email protected]>
15:01:22:      Copyright: 2020 foldingathome.org
15:01:22:       Homepage: https://foldingathome.org/
15:01:22:           Date: Apr 27 2020
15:01:22:           Time: 21:21:01
15:01:22:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
15:01:22:         Branch: master
15:01:22:       Compiler: Visual C++ 2008
15:01:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:01:22:       Platform: win32 10
15:01:22:           Bits: 32
15:01:22:           Mode: Release
15:01:22:         Config: G:\FoldingData\config.xml
15:01:22:******************************** CBang ********************************
15:01:22:           Date: Apr 24 2020
15:01:22:           Time: 17:07:55
15:01:22:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
15:01:22:         Branch: master
15:01:22:       Compiler: Visual C++ 2008
15:01:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:01:22:       Platform: win32 10
15:01:22:           Bits: 32
15:01:22:           Mode: Release
15:01:22:******************************* System ********************************
15:01:22:            CPU: AMD Ryzen 7 2700X Eight-Core Processor
15:01:22:         CPU ID: AuthenticAMD Family 23 Model 8 Stepping 2
15:01:22:           CPUs: 16
15:01:22:         Memory: 31.95GiB
15:01:22:    Free Memory: 23.62GiB
15:01:22:        Threads: WINDOWS_THREADS
15:01:22:     OS Version: 6.2
15:01:22:    Has Battery: false
15:01:22:     On Battery: false
15:01:22:     UTC Offset: -4
15:01:22:            PID: 12488
15:01:22:            CWD: G:\FoldingData
15:01:22:  Win32 Service: false
15:01:22:             OS: Windows 10 Home
15:01:22:        OS Arch: AMD64
15:01:22:           GPUs: 1
15:01:22:          GPU 0: Bus:31 Slot:0 Func:0 AMD:5 Vega 10 XL/XT [Radeon RX Vega 56/64]
15:01:22:           CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll': The
15:01:22:                 specified module could not be found.
15:01:22:
15:01:22:OpenCL Device 0: Platform:0 Device:0 Bus:31 Slot:0 Compute:1.2 Driver:3075.12
15:01:22:******************************* libFAH ********************************
15:01:22:           Date: Apr 15 2020
15:01:22:           Time: 14:53:14
15:01:22:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
15:01:22:         Branch: master
15:01:22:       Compiler: Visual C++ 2008
15:01:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:01:22:       Platform: win32 10
15:01:22:           Bits: 32
15:01:22:           Mode: Release
15:01:22:***********************************************************************
15:01:22:<config>
15:01:22:  <!-- User Information -->
15:01:22:  <passkey v='*****'/>
15:01:22:  <team v='64'/>
15:01:22:  <user v='Crawdaddy79'/>
15:01:22:
15:01:22:  <!-- Folding Slots -->
15:01:22:  <slot id='0' type='CPU'>
15:01:22:    <paused v='true'/>
15:01:22:  </slot>
15:01:22:  <slot id='1' type='GPU'/>
15:01:22:</config>
15:01:22:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:01:23:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:01:23:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:01:23:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:01:23:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:01:24:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:01:24:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:01:24:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:01:24:ERROR:WU00:FS01:Exception: Could not get an assignment
15:01:25:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:01:25:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:01:25:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:01:25:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:01:25:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:01:25:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:01:25:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:01:25:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:01:25:ERROR:WU00:FS01:Exception: Could not get an assignment
15:01:35:FS00:Unpaused
15:01:35:WU01:FS00:Starting
15:01:35:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" G:\FoldingData\cores/cores.foldingathome.org/v7/win/64bit/avx/Core_a7.fah/FahCore_a7.exe -dir 01 -suffix 01 -version 706 -lifeline 12488 -checkpoint 15 -np 14
15:01:35:WU01:FS00:Started FahCore on PID 15680
15:01:35:WU01:FS00:Core PID:9616
15:01:35:WU01:FS00:FahCore 0xa7 started
15:01:35:WU01:FS00:0xa7:*********************** Log Started 2020-06-12T15:01:35Z ***********************
15:01:35:WU01:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
15:01:35:WU01:FS00:0xa7:       Type: 0xa7
15:01:35:WU01:FS00:0xa7:       Core: Gromacs
15:01:35:WU01:FS00:0xa7:       Args: -dir 01 -suffix 01 -version 706 -lifeline 15680 -checkpoint 15 -np
15:01:35:WU01:FS00:0xa7:             14
15:01:35:WU01:FS00:0xa7:************************************ CBang *************************************
15:01:35:WU01:FS00:0xa7:       Date: Oct 26 2019
15:01:35:WU01:FS00:0xa7:       Time: 01:38:25
15:01:35:WU01:FS00:0xa7:   Revision: c46a1a011a24143739ac7218c5a435f66777f62f
15:01:35:WU01:FS00:0xa7:     Branch: master
15:01:35:WU01:FS00:0xa7:   Compiler: Visual C++ 2008
15:01:35:WU01:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:01:35:WU01:FS00:0xa7:   Platform: win32 10
15:01:35:WU01:FS00:0xa7:       Bits: 64
15:01:35:WU01:FS00:0xa7:       Mode: Release
15:01:35:WU01:FS00:0xa7:************************************ System ************************************
15:01:35:WU01:FS00:0xa7:        CPU: AMD Ryzen 7 2700X Eight-Core Processor
15:01:35:WU01:FS00:0xa7:     CPU ID: AuthenticAMD Family 23 Model 8 Stepping 2
15:01:35:WU01:FS00:0xa7:       CPUs: 16
15:01:35:WU01:FS00:0xa7:     Memory: 31.95GiB
15:01:35:WU01:FS00:0xa7:Free Memory: 23.58GiB
15:01:35:WU01:FS00:0xa7:    Threads: WINDOWS_THREADS
15:01:35:WU01:FS00:0xa7: OS Version: 6.2
15:01:35:WU01:FS00:0xa7:Has Battery: false
15:01:35:WU01:FS00:0xa7: On Battery: false
15:01:35:WU01:FS00:0xa7: UTC Offset: -4
15:01:35:WU01:FS00:0xa7:        PID: 9616
15:01:35:WU01:FS00:0xa7:        CWD: G:\FoldingData\work
15:01:35:WU01:FS00:0xa7:******************************** Build - libFAH ********************************
15:01:35:WU01:FS00:0xa7:    Version: 0.0.18
15:01:35:WU01:FS00:0xa7:     Author: Joseph Coffland <[email protected]>
15:01:35:WU01:FS00:0xa7:  Copyright: 2019 foldingathome.org
15:01:35:WU01:FS00:0xa7:   Homepage: https://foldingathome.org/
15:01:35:WU01:FS00:0xa7:       Date: Oct 26 2019
15:01:35:WU01:FS00:0xa7:       Time: 01:52:30
15:01:35:WU01:FS00:0xa7:   Revision: c1e3513b1bc0c16013668f2173ee969e5995b38e
15:01:35:WU01:FS00:0xa7:     Branch: master
15:01:35:WU01:FS00:0xa7:   Compiler: Visual C++ 2008
15:01:35:WU01:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:01:35:WU01:FS00:0xa7:   Platform: win32 10
15:01:35:WU01:FS00:0xa7:       Bits: 64
15:01:35:WU01:FS00:0xa7:       Mode: Release
15:01:35:WU01:FS00:0xa7:************************************ Build *************************************
15:01:35:WU01:FS00:0xa7:       SIMD: avx_256
15:01:35:WU01:FS00:0xa7:********************************************************************************
15:01:35:WU01:FS00:0xa7:Project: 14822 (Run 39, Clone 1, Gen 170)
15:01:35:WU01:FS00:0xa7:Unit: 0x000000c49bf7a4d65ea0a8a59720a4a0
15:01:35:WU01:FS00:0xa7:Digital signatures verified
15:01:35:WU01:FS00:0xa7:Reducing thread count from 14 to 13 to avoid domain decomposition with large prime factor 7
15:01:35:WU01:FS00:0xa7:Reducing thread count from 13 to 12 to avoid domain decomposition by a prime number > 3
15:01:35:WU01:FS00:0xa7:Calling: mdrun -s frame170.tpr -o frame170.trr -cpi state.cpt -cpt 15 -nt 12
15:01:35:WU01:FS00:0xa7:Steps: first=0 total=250000
15:01:36:WU01:FS00:0xa7:Completed 11643 out of 250000 steps (4%)
15:01:56:WU01:FS00:0xa7:Completed 12500 out of 250000 steps (5%)
15:02:22:Removing old file 'configs/config-20200611-024043.xml'
15:02:22:Saving configuration to config.xml
15:02:22:<config>
15:02:22:  <!-- User Information -->
15:02:22:  <passkey v='*****'/>
15:02:22:  <team v='64'/>
15:02:22:  <user v='Crawdaddy79'/>
15:02:22:
15:02:22:  <!-- Folding Slots -->
15:02:22:  <slot id='0' type='CPU'/>
15:02:22:  <slot id='1' type='GPU'/>
15:02:22:</config>
15:02:25:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:02:25:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:02:25:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:02:25:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:02:25:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:02:25:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:02:25:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:02:25:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:02:25:ERROR:WU00:FS01:Exception: Could not get an assignment
15:02:53:WU01:FS00:0xa7:Completed 15000 out of 250000 steps (6%)
15:03:51:WU01:FS00:0xa7:Completed 17500 out of 250000 steps (7%)
15:04:02:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:04:02:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:04:02:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:04:02:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:04:02:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:04:02:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:04:02:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:04:03:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:04:03:ERROR:WU00:FS01:Exception: Could not get an assignment
15:04:49:WU01:FS00:0xa7:Completed 20000 out of 250000 steps (8%)
15:05:47:WU01:FS00:0xa7:Completed 22500 out of 250000 steps (9%)
15:06:39:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:06:39:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:06:39:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:06:39:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:06:39:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:06:39:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:06:39:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:06:40:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:06:40:ERROR:WU00:FS01:Exception: Could not get an assignment
15:06:45:WU01:FS00:0xa7:Completed 25000 out of 250000 steps (10%)
15:07:43:WU01:FS00:0xa7:Completed 27500 out of 250000 steps (11%)
15:08:40:WU01:FS00:0xa7:Completed 30000 out of 250000 steps (12%)
15:09:38:WU01:FS00:0xa7:Completed 32500 out of 250000 steps (13%)
15:10:36:WU01:FS00:0xa7:Completed 35000 out of 250000 steps (14%)
15:10:53:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:10:53:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:10:53:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:10:53:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:10:53:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:10:54:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:10:54:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:10:54:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:10:54:ERROR:WU00:FS01:Exception: Could not get an assignment
15:11:35:WU01:FS00:0xa7:Completed 37500 out of 250000 steps (15%)
15:11:50:FS01:Paused
15:11:55:FS01:Unpaused
15:11:55:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:11:55:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:11:55:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:11:55:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:11:55:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:11:56:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:11:56:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:11:56:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:11:56:ERROR:WU00:FS01:Exception: Could not get an assignment
15:12:33:WU01:FS00:0xa7:Completed 40000 out of 250000 steps (16%)
15:13:31:WU01:FS00:0xa7:Completed 42500 out of 250000 steps (17%)
15:13:32:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:13:32:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:13:32:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:13:33:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:13:33:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:13:33:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
15:13:33:WU00:FS01:Connecting to assign4.foldingathome.org:80
15:13:33:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
15:13:33:ERROR:WU00:FS01:Exception: Could not get an assignment
15:14:29:WU01:FS00:0xa7:Completed 45000 out of 250000 steps (18%)
15:15:27:WU01:FS00:0xa7:Completed 47500 out of 250000 steps (19%)

[ajm]

I have used three GPU folding rigs today. One of them doesn't have any mention of the string "Connecting to assign". The two others do, and most of the time, the next line says "Assigned to work server XXXX" and then comes the usual "Requesting new work unit for slot XX" and so on. But it does happen, like here, that I get "Failed to get assignment from 'assignX.foldingathome.org:80': No WUs available for this configuration".

This looks normal, I'd say. If the AS somehow knows that there's no compatible WU available, it doesn't bother assigning a WS at all.

[Crawdaddy79]

Thanks for the feedback.

Ironically, when I came back to my PC, it was crunching a GPU WU. It grabbed one about two minutes after I posted this thread.

Code: Select all

15:17:18:WU00:FS01:Connecting to assign1.foldingathome.org:80
15:17:18:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:17:18:WU00:FS01:Connecting to assign2.foldingathome.org:80
15:17:19:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
15:17:19:WU00:FS01:Connecting to assign3.foldingathome.org:80
15:17:19:WU00:FS01:Assigned to work server 140.163.4.241
15:17:19:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:Vega 10 XL/XT [Radeon RX Vega 56/64] from 140.163.4.241
15:17:19:WU00:FS01:Connecting to 140.163.4.241:8080
15:17:23:WU01:FS00:0xa7:Completed 52500 out of 250000 steps (21%)
15:17:30:WU00:FS01:Downloading 7.85MiB
15:17:32:WU00:FS01:Download complete
15:17:32:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:11744 run:0 clone:9744 gen:49 core:0x22 unit:0x0000004f8ca304f15e6bc3e66f930bf1
15:17:32:WU00:FS01:Starting

[ajm]

OK - glad it folds again!

[_r2w_ben]

There's less work available for AMD GPUs that are GCN based until the new version of Core 22 is fully tested and generally available.

[foldy]

So maybe it makes sense for @Crawdaddy79 to join the beta team to help testing Core 22 beta on AMD GCN GPU?

[Crawdaddy79]

I'll consider going beta for sure if my system runs dry again. I legitimately thought that because the assignment server change correlated with my lack of WU, that it was the cause. Turns out I just needed to post a thread about it to wake the server up.

EDIT: After finishing that last WU and not getting a new one for five minutes, I added the advanced field to my GPU slot. Then it pulled a 11760 WU.

[bruce]

When the surge of new clients hit FAH a couple months back, all our servers were overwhelmed. We've been addidng server capacity ever since. Even so, we still do get to a point where all servers are running at (or slightly above) reasonable bandwidth and one of the functions of the Assignment servers is to limit the servers to what they can reasonably handle. You get the same message if they can't accept your request or there are no servers that have the kind of assignment your seeking, even if there happen to be assignments for other system configurations than yours. e.g.- it doesn't explicitly say "We have no more GPU WUs right now, would you like a CPU assignment?"

This isn't helped by the testing of new FAHCore when the bug is that your last upload was incomplete so we can't generate the next sequential assignment.