Page 1 of 2
Correct there are not just few but ZERO CPU WUs available?
Posted: Tue Mar 24, 2020 9:05 pm
by Yujah
One of the many new participants here; as such still ogling things...
Although I've been processing some GPU WU's, for the entire day today, 12 hours certainly, I have not received any CPU WUs; the CPU queue just sits idle. Would this still be expected? I read about the shortage but was expecting to get one or two at least; did get some yesterday.
If it helps; my folding-name is also Yujah (currently on team 223518, and passkey'd) and the CPU "Ready" queue lists 128.252.203.10 as the Work Server.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:08 pm
by bruce
I was in a discussion last night discussing the problem of having CPU assignments that are not getting enough people to run them.
Give me details.
EDIT: here are some that came on-line recently. viewtopic.php?f=24&t=33290
I'm sure there are others.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:20 pm
by Neil-B
bruce wrote:I was in a discussion last night discussing the problem of having CPU assignments that are not getting enough people to run them.
Give me details.
EDIT: here are some that came on-line recently. viewtopic.php?f=24&t=33290
I'm sure there are others.
Not intending to highjack thread, but this surprises me a bit … was like the OP just letting my server wait for WUs … It has occasionally picked up on but for the most part has been idle … Running two 28 cpu slots on Neck-W, Team 39363 … If there are CPU assignments not getting done then there is a bottleneck somewhere … btw happy to run more smaller cpu slots if that helps - but is nice to see a 14 day WU crunched in 3/4 hrs.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:25 pm
by Neil-B
bad form reply to self … but then as if by magic two WUs drop in
… You are right they do exist - Thanks.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:27 pm
by Yujah
My CPU's still as idle as it has been all day...
As a newbie I'm relatively uncertain of which exact details to provide (did follow the link in your sig) but studying the log I just now saw that the last CPU WU request in fact showed an explicit error. Only the last, so this may be a red herring, but, with the error message between [ ] manually translated into English from my native language:
===
20:33:00:WU00:FS00:Connecting to 65.254.110.245:8080
20:33:01:WU00:FS00:Assigned to work server 128.252.203.10
20:33:01:WU00:FS00:Requesting new work unit for slot 00: READY cpu:7 from 128.252.203.10
20:33:01:WU00:FS00:Connecting to 128.252.203.10:8080
20:33:18:WU01:FS01:0x22:Completed 520000 out of 1000000 steps (52%)
20:33:22:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
20:33:22:WU00:FS00:Connecting to 128.252.203.10:80
20:33:43:ERROR:WU00:FS00:Exception: Failed to connect to 128.252.203.10:80: [ A connection attempt has failed due to the connected party answering incorrectly after a certain time, or the established connection has failed due to the connected host not answering ]
===
As said, may be a red herring; before there's just many instances of e.g.
===
20:03:58:WU00:FS00:Connecting to 65.254.110.245:8080
20:03:59:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
20:03:59:WU00:FS00:Connecting to 18.218.241.186:80
20:04:05:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:04:05:ERROR:WU00:FS00:Exception: Could not get an assignment
===
Are these the requested details? As said, name Yujah, team 223518; am running GPU WUs --- currently 11762 (0, 3982, 11) --- and the CPU queue is a multi-threaded quad-core, i.e., 8 "CPUs".
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:37 pm
by Wakkaluba
I am facing similar issues.
Normally was at least the GPU working more or less uninterrupted and the CPU slot got 1-2 times a day something to do.
But now? FAH is picking nose since this early morning.
Code: Select all
21:30:57:WU00:FS00:Connecting to 18.218.241.186:80
21:30:57:WU01:FS01:Connecting to 65.254.110.245:8080
21:30:57:WARNING:WU00:FS00:[b]Failed to get assignment[/b] from '18.218.241.186:80': No WUs available for this configuration
21:30:57:ERROR:WU00:FS00:Exception: Could not get an assignment
21:30:57:WARNING:WU01:FS01:[b]Failed to get assignment[/b] from '65.254.110.245:8080': No WUs available for this configuration
21:30:57:WU01:FS01:Connecting to 18.218.241.186:80
21:30:58:WARNING:WU01:FS01:[b]Failed to get assignment[/b] from '18.218.241.186:80': No WUs available for this configuration
21:30:58:ERROR:WU01:FS01:Exception: Could not get an assignment
something that occasionally appeared for the CPU slot also appears for the GPU every now and then..
Code: Select all
21:03:31:WU00:FS00:Connecting to 65.254.110.245:8080
21:03:31:WU01:FS01:Assigned to work server 40.114.52.201
21:03:31:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Tahiti XT [Radeon R9 200/HD 7900/8970] from 40.114.52.201
21:03:31:WU01:FS01:Connecting to 40.114.52.201:8080
21:03:31:WU00:FS00:Assigned to work server 128.252.203.9
21:03:31:WU00:FS00:Requesting new work unit for slot 00: READY cpu:2 from 128.252.203.9
21:03:31:WU00:FS00:Connecting to 128.252.203.9:8080
21:03:43:ERROR:WU01:FS01:Exception: 10002:[b] Received short response, expected 512 bytes, got 0[/b]
21:03:44:ERROR:WU00:FS00:Exception: 10002: [b]Received short response, expected 512 bytes, got 0[/b]
A workserver is assigned but I see the collection server as 0.0.0.0
Not sure if the software fumbles when no collection server is available.
My experience from day one was that something in the infrastructure is causing some race conditions.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 9:43 pm
by Neil-B
Your logs have a very similar pattern to the ones I have been seeing mostly the "No WUs available …" with the occasional nearly - where the request is assigned to a work server but then no response (variety of failure messages) … I have also been seeing an HTTP error at the same type stage iirc … From other posts I had put it down to 1) no WUs (or at least overloaded assignment server) … 2) Overloaded work server not able to service the assignments passed on by the assignment server … 3) network cards struggling with the throughput … Having said that though I am hoping that Bruce is able to spot something that just needs a tweak to keep CPU WU crunchers happy - If it is just heavy load I'll happily leave kit on until such time as it is called on.
Good luck with the CPU WU allocations
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 10:13 pm
by Jesse_V
I'm definitely not Bruce, but I will refer you to a couple of sticky threads at the top of this board: viewtopic.php?f=61&t=33193 and viewtopic.php?f=61&t=32424
The servers are just absolutely flooded with bandwidth and disk I/O. Nobody here really expected a 20x increase in F@h userbase over a few month. A tweak isn't really going to work, but the teams are getting more servers online and more projects into the queue to keep up with the overwhelming demand.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 10:19 pm
by Joe_H
Especially with most of that 20x increase just in the last 10-14 days.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 10:25 pm
by Yujah
As said, I know about the shortage; was/am specifically asking if 0 rather than just few is still expected. And especially in the context of me easily getting CPU WUs only two days ago when I was completely new (and was for a bit folding under a different name then); believe I got 6 or so in a row that first day. it's the 0 rather than few in 12+ hours now which currently has me wonder if I have a local issue.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 10:49 pm
by uyaem
Yujah wrote:As said, I know about the shortage; was/am specifically asking if 0 rather than just few is still expected. And especially in the context of me easily getting CPU WUs only two days ago when I was completely new (and was for a bit folding under a different name then); believe I got 6 or so in a row that first day. it's the 0 rather than few in 12+ hours now which currently has me wonder if I have a local issue.
That's quite normal, I've been getting many GPU ones and few CPU ones since... Saturday.
Also note that the retry timer goes up significantly after each attempt (hours).
As they keep saying, they are working on it
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 10:57 pm
by jonault
Yujah wrote:it's the 0 rather than few in 12+ hours now which currently has me wonder if I have a local issue.
Based on the log file snippets you posted above, you don't have a local issue.
20:03:59:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration means you successfully connected to the work server, it just didn't have anything for you. Nothing you can do except wait for more WUs to become available.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 11:06 pm
by Yujah
Hrmpz. Well, guess I'll exercise patience then.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Tue Mar 24, 2020 11:11 pm
by bruce
jonault wrote:Yujah wrote:it's the 0 rather than few in 12+ hours now which currently has me wonder if I have a local issue.
Based on the log file snippets you posted above, you don't have a local issue.
20:03:59:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration means you successfully connected to the work server, it just didn't have anything for you. Nothing you can do except wait for more WUs to become available.
Not quite. It means you've successfully connected to the Assignment Server, whose job it is to find a Work Server that has WUs for your hardware.
Suppose there are several work servers that do have WUs for you but (A) they're all down for some reason or (B) They all are saturated and currently incapable of accepting any more concurrent requests. It's factually correct that the AS cannot find any Work Server that has work for your configuration but the message doesn't suggest those alternatives.
Actually, (B) is quite common right now. Be patient and sooner or later there will be a gap during which you can get download a WU.
Re: Correct there are not just few but ZERO CPU WUs availabl
Posted: Wed Mar 25, 2020 12:11 am
by bruce
The new servers we brought online yesterday are now completely saturated. The are (rare) breaks when there's room for a new client to get work, but you have to be lucky to hit one of those moments. Be patient. Supply and Demand, where Demand increases at a faster rate than Supply.