Page 1 of 3

Failed to get assignment from '171.67.108.45:80': Empty work

Posted: Sun Mar 04, 2018 11:36 pm
by nsummy

Code: Select all

23:33:16:WU03:FS03:Connecting to 171.67.108.45:80
23:33:17:WARNING:WU03:FS03:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
23:33:17:WU03:FS03:Connecting to 171.64.65.35:80
23:33:17:WU03:FS03:Assigned to work server 171.67.108.157
23:33:17:WU03:FS03:Requesting new work unit for slot 03: READY gpu:3:GP104 [GeForce GTX 1080] 8873 from 171.67.108.157
23:33:18:WU03:FS03:Connecting to 171.67.108.157:8080
23:33:19:WARNING:WU03:FS03:WorkServer connection failed on port 8080 trying 80
23:33:19:WU03:FS03:Connecting to 171.67.108.157:80
23:33:20:ERROR:WU03:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
I'm sure this has been asked and answered a million times, but is there a way to force it to failover to another server? Seems odd that the fah software would repeatedly try to connect to the same server that fails.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:00 am
by SteveWillis
I'm having a similar problem

Code: Select all

2018-03-04:23:54:22:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:54:22:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:54:23:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:54:23:WARNING:WU01:FS02:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:54:24:ERROR:WU00:FS00:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:54:24:ERROR:WU01:FS02:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:54:25:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:54:25:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:54:25:WARNING:WU01:FS02:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:54:25:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:54:25:ERROR:WU01:FS02:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:54:26:ERROR:WU00:FS00:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:55:25:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:55:25:WARNING:WU01:FS02:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:55:26:ERROR:WU01:FS02:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:55:34:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:55:35:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:55:36:ERROR:WU00:FS00:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:56:25:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:56:26:ERROR:WU01:FS02:Exception: Server did not assign work unit
2018-03-04:23:56:57:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:56:59:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:56:59:ERROR:WU00:FS00:Exception: Failed to connect to 171.67.108.157:80: Connection refused
2018-03-04:23:57:25:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
2018-03-04:23:57:26:WARNING:WU01:FS02:WorkServer connection failed on port 8080 trying 80
2018-03-04:23:57:26:ERROR:WU01:FS02:Exception: Failed to connect to 171.67.108.157:80: Connection refused

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:18 am
by chris21010

Code: Select all

23:13:21:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:13:21:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:13:23:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:13:23:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:13:23:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:13:23:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:13:25:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:13:25:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:13:26:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:13:26:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:14:05:WARNING:FS03:Size of positions 394 does not match topology 391
23:14:20:WARNING:Exception: boost::filesystem::remove: Access is denied: "configs/config-20161017-004911.xml"
23:14:23:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:14:24:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:14:25:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:14:25:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:14:26:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:14:27:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:14:41:WARNING:FS03:Size of positions 394 does not match topology 391
23:15:18:WARNING:FS03:Size of positions 394 does not match topology 391
23:15:55:WARNING:FS03:Size of positions 394 does not match topology 391
23:16:00:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:16:01:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:16:02:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:16:02:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:16:03:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:16:16:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
23:16:32:WARNING:FS03:Size of positions 394 does not match topology 391
23:17:09:WARNING:FS03:Size of positions 394 does not match topology 391
23:17:46:WARNING:FS03:Size of positions 394 does not match topology 391
23:18:23:WARNING:FS03:Size of positions 394 does not match topology 391
23:18:37:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:18:38:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:18:39:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:18:40:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:18:40:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:18:41:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:19:00:WARNING:FS03:Size of positions 394 does not match topology 391
23:19:37:WARNING:FS03:Size of positions 394 does not match topology 391
23:20:14:WARNING:FS03:Size of positions 394 does not match topology 391
23:20:51:WARNING:FS03:Size of positions 394 does not match topology 391
23:21:28:WARNING:FS03:Size of positions 394 does not match topology 391
23:22:05:WARNING:FS03:Size of positions 394 does not match topology 391
23:22:42:WARNING:FS03:Size of positions 394 does not match topology 391
23:22:47:WARNING:FS01:Size of positions 4527 does not match topology 2736
23:22:52:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:22:52:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:22:53:WARNING:FS05:Size of positions 5091 does not match topology 2736
23:22:54:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:22:54:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:22:55:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:22:55:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:23:20:WARNING:FS03:Size of positions 394 does not match topology 391
23:23:57:WARNING:FS03:Size of positions 394 does not match topology 391
23:24:34:WARNING:FS03:Size of positions 394 does not match topology 391
23:25:11:WARNING:FS03:Size of positions 394 does not match topology 391
23:25:48:WARNING:FS03:Size of positions 394 does not match topology 391
23:26:12:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:26:15:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:26:16:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:26:17:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:26:18:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:26:20:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:26:25:WARNING:FS03:Size of positions 394 does not match topology 391
23:26:59:WARNING:WU01:FS03:WorkServer connection failed on port 8080 trying 80
23:27:00:WARNING:WU01:FS03:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:27:17:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:27:18:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:27:20:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:28:54:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:28:56:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:28:57:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:29:43:ERROR:WU02:FS02:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
23:29:43:ERROR:WU02:FS02:Exception: Could not get IP address for assign-GPU2.stanford.edu: No such host is known. 
23:29:43:WARNING:WU02:FS02:Exception: Failed to find any IP addresses for assignment servers
23:29:43:ERROR:WU02:FS02:Exception: Could not get an assignment
23:29:43:ERROR:WU03:FS04:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
23:29:43:ERROR:WU03:FS04:Exception: Could not get IP address for assign-GPU2.stanford.edu: No such host is known. 
23:29:43:WARNING:WU03:FS04:Exception: Failed to find any IP addresses for assignment servers
23:29:43:ERROR:WU03:FS04:Exception: Could not get an assignment
23:31:31:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:31:33:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:31:34:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:31:58:WARNING:FS01:Size of positions 4527 does not match topology 2736
23:32:12:WARNING:FS05:Size of positions 5091 does not match topology 2736
23:35:45:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:35:47:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:35:49:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:38:58:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:39:00:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:39:01:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:39:02:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:39:04:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:39:05:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:40:02:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:40:04:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:40:05:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:40:49:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:40:49:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:40:51:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:40:51:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:40:52:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:40:52:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:41:29:WARNING:FS05:Size of positions 5091 does not match topology 2736
23:41:40:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:41:41:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:41:43:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:42:37:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:42:39:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:42:40:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:44:16:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:44:18:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:44:20:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:48:31:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:48:33:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:48:34:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:50:45:WARNING:FS05:Size of positions 5091 does not match topology 2736
23:53:42:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:53:44:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
23:53:45:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:55:22:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:55:24:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:55:25:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:58:46:WARNING:WU02:FS02:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:58:46:WARNING:WU03:FS04:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
23:58:48:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
23:58:48:WARNING:WU03:FS04:WorkServer connection failed on port 8080 trying 80
23:58:49:ERROR:WU02:FS02:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
23:58:50:ERROR:WU03:FS04:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
00:00:02:WARNING:FS05:Size of positions 5091 does not match topology 2736
00:06:28:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
00:06:30:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:06:31:ERROR:WU01:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
00:09:19:WARNING:FS05:Size of positions 5091 does not match topology 2736
00:11:39:WARNING:WU04:FS03:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
00:11:41:WARNING:WU04:FS03:WorkServer connection failed on port 8080 trying 80
00:11:42:ERROR:WU04:FS03:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
dito, couple hours now. tried reboot and restarting app to no avail.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:35 am
by nsummy
I tried deleting the GPU and re-adding it, hoping it would grab a different server, but it went back to the same one :(

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:37 am
by Aurum
Same. Rebooting didn't help.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:46 am
by Hypocritus
Can confirm that I am not receiving any WUs.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 12:58 am
by Aurum
140.163.4.232 is still DLing. During a previous WS failure someone posted these rules:
netsh advfirewall firewall add rule name="FahClient workaround" dir=in interface=any action=block remoteip=171.67.108.105
netsh advfirewall firewall add rule name="FahClient workaround" dir=in interface=any action=block remoteip=140.163.4.241
netsh advfirewall firewall delete rule name="FahClient workaround"

This was for Windows and I didn't copy the Linux approach. Might help. Also, if you switch to a different cause it might redirect as well.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:02 am
by Aurum
171.64.65.35 serving no WUs either.
I tried switching from Alzheimer's to Parkinson's and reloading FAH but still nothing.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:07 am
by Aurum
Looks wide spread and on a Sunday so it's time to go help Einstein discover pulsars and gravity waves.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:10 am
by SteveWillis
All my FSs are working now. My script does a pause/unpause once a minute and after about an hour it worked. I had 5 FSs that were down at one point but now all are running.
Current version of the script is at https://drive.google.com/drive/folders/ ... sp=sharing but it's not totally debugged.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:14 am
by nsummy
Just finished another WU, and still not downloading a new one, this time it shows 0.0.0.0 for the address, so maybe there is a larger issue here

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:20 am
by Aurum
140.163.4.231 is serving WUs.

Steve that's a serious script.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 1:31 am
by SteveWillis
:-) I've been working on it a while.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 2:10 am
by Joe_H
The WS 171.67.108.157 is showing as having gone into Standby status on the Server Status page. I have notified the owner of that server. I have also asked that the Assignment Servers be checked on, they should have switched assigning to other available Work Servers.

P.S. Pushing connections to 140.163.4.231 and its fellow WS's will have limited use, they do not have a large number of available WU's to replace the supply of the WS that is down.

Re: Failed to get assignment from '171.67.108.45:80': Empty

Posted: Mon Mar 05, 2018 3:17 am
by Hypocritus
From FAHControl, select "Configure", navigate to the last tab "Expert" and under "Extra client options",
looks like setting

Code: Select all

client-type
advanced
is a way to get WUs within one or two attempts.