Page 5 of 7
Re: Can't upload to 140.163.4.231 again
Posted: Tue Apr 14, 2020 3:11 pm
by alxbelu
vmzy wrote:bruce wrote:The PLEASE_WAIT issue is a known problem. FAHClient should retry the upload again If, after a few tries, it still has not been accepted, please add that information below.
it has been retry over 15 times in these two days.
Not sure how many times "a few times" are, but I'm up to 7 attempts now, with all uploads completing followed by the PLEASE_WAIT response, and considering the fact that the upload takes around 50-180 seconds (3.89-1.08Mbps) on my 100/100Mbps connection, I'm guessing the WS network connection is getting saturated by all the retries that are allowed to complete (but forced to retry).
Code: Select all
14:47:20:WU01:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
14:47:26:WU01:FS01:0x22:Saving result file ..\logfile_01.txt
14:47:26:WU01:FS01:0x22:Saving result file checkpointState.xml
14:47:31:WU01:FS01:0x22:Saving result file checkpt.crc
14:47:31:WU01:FS01:0x22:Saving result file positions.xtc
14:47:33:WU01:FS01:0x22:Saving result file science.log
14:47:33:WU01:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
14:47:34:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
14:47:34:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:47:34:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:47:34:WU01:FS01:Connecting to 140.163.4.231:8080
14:47:40:WU01:FS01:Upload 12.07%
14:47:46:WU01:FS01:Upload 25.68%
14:47:52:WU01:FS01:Upload 39.03%
14:47:58:WU01:FS01:Upload 52.13%
14:48:04:WU01:FS01:Upload 64.96%
14:48:10:WU01:FS01:Upload 78.32%
14:48:16:WU01:FS01:Upload 91.67%
14:48:20:WU01:FS01:Upload complete
14:48:20:WU01:FS01:Server responded PLEASE_WAIT (464)
14:48:20:WARNING:WU01:FS01:Failed to send results, will try again later
14:48:20:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:48:20:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:48:20:WU01:FS01:Connecting to 140.163.4.231:8080
14:48:26:WU01:FS01:Upload 10.53%
14:48:32:WU01:FS01:Upload 20.54%
14:48:38:WU01:FS01:Upload 32.61%
14:48:44:WU01:FS01:Upload 44.17%
14:48:50:WU01:FS01:Upload 56.75%
14:48:56:WU01:FS01:Upload 66.25%
14:49:02:WU01:FS01:Upload 77.55%
14:49:08:WU01:FS01:Upload 89.36%
14:49:14:WU01:FS01:Upload 99.63%
14:49:14:WU01:FS01:Upload complete
14:49:14:WU01:FS01:Server responded PLEASE_WAIT (464)
14:49:14:WARNING:WU01:FS01:Failed to send results, will try again later
14:49:20:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:49:20:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:49:20:WU01:FS01:Connecting to 140.163.4.231:8080
14:49:26:WU01:FS01:Upload 3.59%
14:49:32:WU01:FS01:Upload 7.70%
14:49:38:WU01:FS01:Upload 12.07%
14:49:44:WU01:FS01:Upload 16.43%
14:49:50:WU01:FS01:Upload 20.54%
14:49:56:WU01:FS01:Upload 24.65%
14:50:02:WU01:FS01:Upload 28.50%
14:50:08:WU01:FS01:Upload 32.61%
14:50:14:WU01:FS01:Upload 36.46%
14:50:20:WU01:FS01:Upload 40.31%
14:50:26:WU01:FS01:Upload 44.17%
14:50:32:WU01:FS01:Upload 48.02%
14:50:38:WU01:FS01:Upload 51.87%
14:50:44:WU01:FS01:Upload 55.72%
14:50:50:WU01:FS01:Upload 59.57%
14:50:56:WU01:FS01:Upload 63.42%
14:51:02:WU01:FS01:Upload 67.27%
14:51:08:WU01:FS01:Upload 71.13%
14:51:14:WU01:FS01:Upload 74.72%
14:51:20:WU01:FS01:Upload 78.57%
14:51:26:WU01:FS01:Upload 82.42%
14:51:32:WU01:FS01:Upload 86.28%
14:51:38:WU01:FS01:Upload 89.61%
14:51:44:WU01:FS01:Upload 93.47%
14:51:50:WU01:FS01:Upload 97.32%
14:51:54:WU01:FS01:Upload complete
14:51:54:WU01:FS01:Server responded PLEASE_WAIT (464)
14:51:54:WARNING:WU01:FS01:Failed to send results, will try again later
14:51:55:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:51:55:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:51:55:WU01:FS01:Connecting to 140.163.4.231:8080
14:52:01:WU01:FS01:Upload 12.07%
14:52:07:WU01:FS01:Upload 25.42%
14:52:13:WU01:FS01:Upload 39.03%
14:52:19:WU01:FS01:Upload 50.84%
14:52:25:WU01:FS01:Upload 60.34%
14:52:31:WU01:FS01:Upload 72.67%
14:52:37:WU01:FS01:Upload 86.02%
14:52:43:WU01:FS01:Upload 98.86%
14:52:43:WU01:FS01:Upload complete
14:52:43:WU01:FS01:Server responded PLEASE_WAIT (464)
14:52:43:WARNING:WU01:FS01:Failed to send results, will try again later
14:54:32:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:54:32:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:54:32:WU01:FS01:Connecting to 140.163.4.231:8080
14:54:38:WU01:FS01:Upload 6.16%
14:54:44:WU01:FS01:Upload 13.35%
14:54:50:WU01:FS01:Upload 20.03%
14:54:56:WU01:FS01:Upload 26.70%
14:55:02:WU01:FS01:Upload 33.38%
14:55:08:WU01:FS01:Upload 40.57%
14:55:14:WU01:FS01:Upload 46.73%
14:55:20:WU01:FS01:Upload 49.81%
14:55:26:WU01:FS01:Upload 56.75%
14:55:32:WU01:FS01:Upload 63.42%
14:55:38:WU01:FS01:Upload 69.84%
14:55:44:WU01:FS01:Upload 76.01%
14:55:50:WU01:FS01:Upload 81.40%
14:55:56:WU01:FS01:Upload 86.02%
14:56:02:WU01:FS01:Upload 92.18%
14:56:08:WU01:FS01:Upload 98.34%
14:56:10:WU01:FS01:Upload complete
14:56:10:WU01:FS01:Server responded PLEASE_WAIT (464)
14:56:10:WARNING:WU01:FS01:Failed to send results, will try again later
14:58:46:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
14:58:46:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
14:58:46:WU01:FS01:Connecting to 140.163.4.231:8080
14:58:52:WU01:FS01:Upload 10.53%
14:58:58:WU01:FS01:Upload 21.31%
14:59:04:WU01:FS01:Upload 31.84%
14:59:11:WU01:FS01:Upload 42.11%
14:59:17:WU01:FS01:Upload 51.61%
14:59:23:WU01:FS01:Upload 60.86%
14:59:29:WU01:FS01:Upload 74.21%
14:59:35:WU01:FS01:Upload 87.82%
14:59:41:WU01:FS01:Upload 99.89%
14:59:41:WU01:FS01:Upload complete
14:59:41:WU01:FS01:Server responded PLEASE_WAIT (464)
14:59:41:WARNING:WU01:FS01:Failed to send results, will try again later
15:05:37:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11752 run:0 clone:7068 gen:2 core:0x22 unit:0x000000098ca304e75e6bbebd1ad2e524
15:05:37:WU01:FS01:Uploading 24.34MiB to 140.163.4.231
15:05:37:WU01:FS01:Connecting to 140.163.4.231:8080
15:05:44:WU01:FS01:Upload 2.05%
15:05:50:WU01:FS01:Upload 4.88%
15:05:56:WU01:FS01:Upload 6.93%
15:06:02:WU01:FS01:Upload 11.30%
15:06:08:WU01:FS01:Upload 14.89%
15:06:14:WU01:FS01:Upload 17.97%
15:06:20:WU01:FS01:Upload 21.57%
15:06:26:WU01:FS01:Upload 23.62%
15:06:32:WU01:FS01:Upload 26.45%
15:06:38:WU01:FS01:Upload 29.27%
15:06:44:WU01:FS01:Upload 32.87%
15:06:50:WU01:FS01:Upload 37.23%
15:06:56:WU01:FS01:Upload 39.03%
15:07:02:WU01:FS01:Upload 41.85%
15:07:11:WU01:FS01:Upload 44.42%
15:07:17:WU01:FS01:Upload 48.79%
15:07:23:WU01:FS01:Upload 52.38%
15:07:29:WU01:FS01:Upload 55.46%
15:07:35:WU01:FS01:Upload 59.57%
15:07:41:WU01:FS01:Upload 63.42%
15:07:47:WU01:FS01:Upload 65.73%
15:07:53:WU01:FS01:Upload 69.84%
15:07:59:WU01:FS01:Upload 73.69%
15:08:05:WU01:FS01:Upload 78.32%
15:08:11:WU01:FS01:Upload 82.94%
15:08:17:WU01:FS01:Upload 87.30%
15:08:23:WU01:FS01:Upload 91.93%
15:08:29:WU01:FS01:Upload 96.80%
15:08:35:WU01:FS01:Upload complete
15:08:35:WU01:FS01:Server responded PLEASE_WAIT (464)
15:08:35:WARNING:WU01:FS01:Failed to send results, will try again later
Re: Can't upload to 140.163.4.231 again
Posted: Tue Apr 14, 2020 7:17 pm
by uyaem
Looks like this is a possible enhancement for the next client version: A better back-off strategy when PLEASE_WAIT is encountered.
After the first few server responses, the client tries again immediately, which will not be healthy for the server.
Then again once there is more hardware set up, maybe this will become a non-issue.
Stuck on "Send Results" with 42 attempts already
Posted: Tue Apr 14, 2020 8:20 pm
by hormelspam552
One of my completed WUs has been stuck on "Send Results" with 42 attempts already, causing it to lose over 80% of its credit value. Any way to fix this client-side or is it a problem with the server?
Here's a snippet of what the logs say:
Code: Select all
20:05:39:WU01:FS01:Upload 57.56%
20:05:45:WU01:FS01:Upload 75.04%
20:05:51:WU01:FS01:Upload complete
20:05:51:WU01:FS01:Server responded PLEASE_WAIT (464)
20:05:51:WARNING:WU01:FS01:Failed to send results, will try again later
20:15:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:7563 gen:17 core:0x22 unit:0x0000001d8ca304e75e6bbc3ddad0f431
20:15:25:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
20:15:25:WU01:FS01:Connecting to 140.163.4.231:8080
20:15:31:WU01:FS01:Upload 21.74%
20:15:37:WU01:FS01:Upload 47.75%
20:15:43:WU01:FS01:Upload 73.76%
20:15:49:WU01:FS01:Upload 92.52%
20:15:51:WU01:FS01:Upload complete
20:15:51:WU01:FS01:Server responded PLEASE_WAIT (464)
20:15:51:WARNING:WU01:FS01:Failed to send results, will try again later
20:17:03:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:7563 gen:17 core:0x22 unit:0x0000001d8ca304e75e6bbc3ddad0f431
Re: Stuck on "Send Results" with 42 attempts already
Posted: Tue Apr 14, 2020 8:27 pm
by uyaem
The
stats page is reporting the server in question (140.163.4.231) as up with ~15 TB of space available as of 15min ago.
In theory this server has a collection server (CS, see "Has CS" column) as backup for offloading results in case that server is too busy/full.
I'm only replying to say: There is nothing you can do, except what you did by posting this.
Maybe one of the moderators has the means to pick this up and contact the server responsible.
Re: Can't upload to 140.163.4.231 again
Posted: Tue Apr 14, 2020 9:21 pm
by Havrisone
same issue here
PRCG 11750(0,2179,4)
12 upload attempts failed already in a row on 'Server responded PLEASE_WAIT (464)'
Re: Can't upload to 140.163.4.231 again
Posted: Tue Apr 14, 2020 11:51 pm
by JohnChodera
We've restarted plfah1-1.mskcc.org (140.163.4.231) after it was having some file locking issues. The logs seem to be happier now, so please let us know if there are still issues!
Thanks for bearing with us!
Re: Can't upload to 140.163.4.231 again
Posted: Wed Apr 15, 2020 11:43 am
by alxbelu
JohnChodera wrote:We've restarted plfah1-1.mskcc.org (140.163.4.231) after it was having some file locking issues. The logs seem to be happier now, so please let us know if there are still issues!
Thanks for bearing with us!
Yep, seems to have worked, my WU was successfully uploaded - thank you!
Re: Can't upload to 140.163.4.231 again
Posted: Thu Apr 16, 2020 5:44 am
by paul_k
Code: Select all
05:24:49:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
05:24:49:WU01:FS01:Connecting to 140.163.4.231:8080
05:24:55:WU01:FS01:Upload 17.48%
05:25:01:WU01:FS01:Upload 36.23%
05:25:07:WU01:FS01:Upload 55.84%
05:25:13:WU01:FS01:Upload 69.48%
05:25:19:WU01:FS01:Upload 88.23%
05:25:22:WU01:FS01:Upload complete
05:25:22:WU01:FS01:Server responded PLEASE_WAIT (464)
05:25:22:WARNING:WU01:FS01:Failed to send results, will try again later
05:25:49:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:2222 gen:10 core:0x22 unit:0x000000158ca304e75e6a8045b56e04fb
05:25:49:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
05:25:49:WU01:FS01:Connecting to 140.163.4.231:8080
05:25:55:WU01:FS01:Upload 14.49%
05:26:01:WU01:FS01:Upload 34.10%
05:26:07:WU01:FS01:Upload 53.28%
05:26:13:WU01:FS01:Upload 68.63%
05:26:15:WU02:FS01:0x22:Completed 560000 out of 1000000 steps (56%)
05:26:19:WU01:FS01:Upload 85.68%
05:26:25:WU01:FS01:Upload 99.74%
05:26:25:WU01:FS01:Upload complete
05:26:25:WU01:FS01:Server responded PLEASE_WAIT (464)
05:26:25:WARNING:WU01:FS01:Failed to send results, will try again later
05:27:26:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:2222 gen:10 core:0x22 unit:0x000000158ca304e75e6a8045b56e04fb
05:27:26:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
05:27:26:WU01:FS01:Connecting to 140.163.4.231:8080
05:27:32:WU01:FS01:Upload 15.77%
05:27:38:WU01:FS01:Upload 35.38%
05:27:44:WU01:FS01:Upload 49.87%
05:27:51:WU01:FS01:Upload 59.68%
05:27:57:WU01:FS01:Upload 75.45%
05:28:04:WU01:FS01:Upload 93.35%
05:28:06:WU02:FS01:0x22:Completed 570000 out of 1000000 steps (57%)
05:28:08:WU01:FS01:Upload complete
05:28:08:WU01:FS01:Server responded PLEASE_WAIT (464)
05:28:08:WARNING:WU01:FS01:Failed to send results, will try again later
05:30:03:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:2222 gen:10 core:0x22 unit:0x000000158ca304e75e6a8045b56e04fb
05:30:03:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
05:30:03:WU01:FS01:Connecting to 140.163.4.231:8080
05:30:09:WU01:FS01:Upload 10.23%
05:30:16:WU01:FS01:Upload 20.03%
05:30:22:WU01:FS01:Upload 36.66%
05:30:25:WU02:FS01:0x22:Completed 580000 out of 1000000 steps (58%)
05:30:28:WU01:FS01:Upload 56.27%
05:30:34:WU01:FS01:Upload 68.20%
05:30:40:WU01:FS01:Upload 83.12%
05:30:45:WU01:FS01:Upload complete
05:30:45:WU01:FS01:Server responded PLEASE_WAIT (464)
05:30:45:WARNING:WU01:FS01:Failed to send results, will try again later
05:32:44:WU02:FS01:0x22:Completed 590000 out of 1000000 steps (59%)
05:34:17:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11751 run:0 clone:2222 gen:10 core:0x22 unit:0x000000158ca304e75e6a8045b56e04fb
05:34:17:WU01:FS01:Uploading 14.66MiB to 140.163.4.231
05:34:17:WU01:FS01:Connecting to 140.163.4.231:8080
05:34:27:WU01:FS01:Upload 1.28%
05:34:33:WU01:FS01:Upload 19.61%
05:34:39:WU01:FS01:Upload 30.69%
05:34:50:WU01:FS01:Upload 44.33%
05:34:56:WU01:FS01:Upload 52.00%
05:35:02:WU01:FS01:Upload 69.05%
05:35:06:WU02:FS01:0x22:Completed 600000 out of 1000000 steps (60%)
05:35:08:WU01:FS01:Upload 77.58%
05:35:14:WU01:FS01:Upload 79.71%
05:35:20:WU01:FS01:Upload 95.48%
05:35:26:WU01:FS01:Upload 100.00%
05:35:26:WU01:FS01:Upload complete
05:35:26:WU01:FS01:Server responded PLEASE_WAIT (464)
05:35:26:WARNING:WU01:FS01:Failed to send results, will try again later
Can't upload.
Re: Can't upload to 140.163.4.231 again
Posted: Thu Apr 16, 2020 1:44 pm
by HaloJones
same here
Re: Can't upload to 140.163.4.231 again
Posted: Thu Apr 16, 2020 6:41 pm
by HaloJones
Interesting. From when this unit finished and failed to upload I have been unable to either upload it or get a new unit to work on. I have just stopped the client, renamed the work directory to wok.old and rebooted and instantly got more work.
The loss of points through not folding was much higher than the (decreasing) loss of the points on the completed unit.
Re: Can't upload to 140.163.4.231 again
Posted: Thu Apr 16, 2020 9:55 pm
by NuovaApe
JohnChodera wrote:We've restarted plfah1-1.mskcc.org (140.163.4.231) after it was having some file locking issues. The logs seem to be happier now, so please let us know if there are still issues!
Thanks for bearing with us!
Number of times I've seen Anti Virus scanners causing this.
Re: Can't upload to 140.163.4.231 again
Posted: Fri Apr 17, 2020 12:02 pm
by NathanJanssens
Been having issues here for a day or 2 now.
I would like to know if there's a way to delete the finished work if the deadline has passed. There is no sense in putting load on the server to send work that will not be taken into account, as I understand it.
Here the logs:
Config
Code: Select all
*********************** Log Started 2020-04-17T08:04:16Z ***********************
08:04:16:************************* Folding@home Client *************************
08:04:16: Website: https://foldingathome.org/
08:04:16: Copyright: (c) 2009-2018 foldingathome.org
08:04:16: Author: Joseph Coffland <[email protected]>
08:04:16: Args:
08:04:16: Config: ********************
08:04:16:******************************** Build ********************************
08:04:16: Version: 7.5.1
08:04:16: Date: May 11 2018
08:04:16: Time: 13:06:32
08:04:16: Repository: Git
08:04:16: Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
08:04:16: Branch: master
08:04:16: Compiler: Visual C++ 2008
08:04:16: Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
08:04:16: Platform: win32 10
08:04:16: Bits: 32
08:04:16: Mode: Release
08:04:16:******************************* System ********************************
08:04:16: CPU: AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx
08:04:16: CPU ID: AuthenticAMD Family 23 Model 24 Stepping 1
08:04:16: CPUs: 8
08:04:16: Memory: 13.92GiB
08:04:16: Free Memory: 11.23GiB
08:04:16: Threads: WINDOWS_THREADS
08:04:16: OS Version: 6.2
08:04:16: Has Battery: true
08:04:16: On Battery: false
08:04:16: UTC Offset: 2
08:04:16: PID: 10720
08:04:16: CWD: *****************
08:04:16: OS: Windows 10 Home
08:04:16: OS Arch: AMD64
08:04:16: GPUs: 2
08:04:16: GPU 0: Bus:3 Slot:0 Func:0 AMD:6 Navi 14 [Radeon RX 5500/5500M / Pro
08:04:16: 5500M]
08:04:16: GPU 1: Bus:7 Slot:0 Func:0 AMD:5 Radeon RX Vega 10 Mobile
08:04:16: CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll': The
08:04:16: specified module could not be found.
08:04:16:
08:04:16:OpenCL Device 0: Platform:0 Device:0 Bus:3 Slot:0 Compute:1.2 Driver:2906.10
08:04:16:OpenCL Device 1: Platform:0 Device:1 Bus:7 Slot:0 Compute:1.2 Driver:2906.10
08:04:16: Win32 Service: false
08:04:16:***********************************************************************
08:04:16:<config>
08:04:16: <!-- Folding Core -->
08:04:16: <checkpoint v='6'/>
08:04:16:
08:04:16: <!-- Network -->
08:04:16: <proxy v=':8080'/>
08:04:16:
08:04:16: <!-- Slot Control -->
08:04:16: <power v='full'/>
08:04:16:
08:04:16: <!-- User Information -->
08:04:16: <passkey v='********************************'/>
08:04:16: <user v='NathanJanssens'/>
08:04:16:
08:04:16: <!-- Folding Slots -->
08:04:16: <slot id='0' type='CPU'/>
08:04:16: <slot id='1' type='GPU'/>
08:04:16: <slot id='2' type='GPU'/>
08:04:16:</config>
Code: Select all
08:05:49:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
08:05:49:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
08:05:49:WU01:FS02:Connecting to 140.163.4.231:8080
08:05:58:WU01:FS02:Upload 0.99%
08:06:04:WU01:FS02:Upload 8.46%
08:06:10:WU01:FS02:Upload 13.93%
08:06:16:WU01:FS02:Upload 22.38%
08:06:22:WU01:FS02:Upload 38.79%
08:06:28:WU01:FS02:Upload 48.74%
08:06:40:WU01:FS02:Upload 59.68%
08:06:46:WU01:FS02:Upload 62.67%
08:06:52:WU01:FS02:Upload 66.15%
08:06:58:WU01:FS02:Upload 71.12%
08:07:04:WU01:FS02:Upload 77.09%
08:07:10:WU01:FS02:Upload 93.51%
08:07:12:WU01:FS02:Upload complete
08:07:12:WU01:FS02:Server responded PLEASE_WAIT (464)
08:07:12:WARNING:WU01:FS02:Failed to send results, will try again later
08:07:12:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
08:07:12:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
08:07:12:WU01:FS02:Connecting to 140.163.4.231:8080
08:07:18:WU01:FS02:Upload 20.89%
08:07:25:WU01:FS02:Upload 35.81%
08:07:31:WU01:FS02:Upload 57.20%
08:07:37:WU01:FS02:Upload 62.67%
08:07:43:WU01:FS02:Upload 66.65%
08:07:49:WU01:FS02:Upload 74.61%
08:07:55:WU01:FS02:Upload 93.01%
08:07:56:WU01:FS02:Upload complete
08:07:57:WU01:FS02:Server responded PLEASE_WAIT (464)
08:07:57:WARNING:WU01:FS02:Failed to send results, will try again later
08:08:49:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
08:08:49:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
08:08:49:WU01:FS02:Connecting to 140.163.4.231:8080
08:08:55:WU01:FS02:Upload 9.95%
08:09:01:WU01:FS02:Upload 13.43%
08:09:07:WU01:FS02:Upload 22.88%
08:09:13:WU01:FS02:Upload 42.77%
08:09:19:WU01:FS02:Upload 63.66%
08:09:25:WU01:FS02:Upload 80.57%
08:09:30:WU01:FS02:Upload complete
08:09:30:WU01:FS02:Server responded PLEASE_WAIT (464)
08:09:30:WARNING:WU01:FS02:Failed to send results, will try again later
........
09:20:37:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
09:20:37:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
09:20:37:WU01:FS02:Connecting to 140.163.4.231:8080
09:21:04:WU01:FS02:Upload 0.99%
09:21:10:WU01:FS02:Upload 7.96%
09:21:16:WU01:FS02:Upload 14.42%
09:21:22:WU01:FS02:Upload 25.86%
09:21:28:WU01:FS02:Upload 44.27%
09:21:34:WU01:FS02:Upload 64.16%
09:21:40:WU01:FS02:Upload 77.09%
09:21:46:WU01:FS02:Upload 92.01%
09:21:52:WU01:FS02:Upload 96.49%
09:21:54:WU01:FS02:Upload complete
09:21:54:WU01:FS02:Server responded PLEASE_WAIT (464)
09:21:54:WARNING:WU01:FS02:Failed to send results, will try again later
10:07:35:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
10:07:35:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
10:07:35:WU01:FS02:Connecting to 140.163.4.231:8080
10:07:47:WU01:FS02:Upload 0.99%
10:07:53:WU01:FS02:Upload 4.97%
10:07:59:WU01:FS02:Upload 8.46%
10:08:05:WU01:FS02:Upload 20.39%
10:08:11:WU01:FS02:Upload 29.34%
10:08:17:WU01:FS02:Upload 38.79%
10:08:23:WU01:FS02:Upload 48.74%
10:08:29:WU01:FS02:Upload 60.68%
10:08:35:WU01:FS02:Upload 68.14%
10:08:43:WU01:FS02:Upload 72.12%
10:08:49:WU01:FS02:Upload 77.59%
10:08:55:WU01:FS02:Upload 99.97%
10:08:56:WU01:FS02:Upload complete
10:08:56:WU01:FS02:Server responded PLEASE_WAIT (464)
10:08:56:WARNING:WU01:FS02:Failed to send results, will try again later
11:23:36:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
11:23:36:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
11:23:36:WU01:FS02:Connecting to 140.163.4.231:8080
11:24:03:WU01:FS02:Upload 0.99%
11:24:09:WU01:FS02:Upload 7.96%
11:24:15:WU01:FS02:Upload 26.36%
11:24:21:WU01:FS02:Upload 50.23%
11:24:27:WU01:FS02:Upload 68.64%
11:24:33:WU01:FS02:Upload 93.51%
11:24:39:WU01:FS02:Upload complete
11:24:39:WU01:FS02:Server responded PLEASE_WAIT (464)
11:24:39:WARNING:WU01:FS02:Failed to send results, will try again later
Code: Select all
08:54:35:WU02:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
08:54:47:WU02:FS01:0x22:Saving result file ..\logfile_01.txt
08:54:47:WU02:FS01:0x22:Saving result file checkpointState.xml
08:54:55:WU02:FS01:0x22:Saving result file checkpt.crc
08:54:55:WU02:FS01:0x22:Saving result file positions.xtc
08:55:01:WU02:FS01:0x22:Saving result file science.log
08:55:01:WU02:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
08:55:02:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:55:02:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
08:55:02:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
08:55:02:WU02:FS01:Connecting to 140.163.4.231:8080
08:55:31:WU02:FS01:Upload 0.51%
08:55:37:WU02:FS01:Upload 6.42%
08:55:43:WU02:FS01:Upload 11.04%
08:55:49:WU02:FS01:Upload 13.87%
08:55:55:WU02:FS01:Upload 17.72%
08:56:01:WU02:FS01:Upload 28.77%
08:56:07:WU02:FS01:Upload 39.81%
08:56:13:WU02:FS01:Upload 43.15%
08:56:21:WU02:FS01:Upload 51.62%
08:56:27:WU02:FS01:Upload 54.71%
08:56:33:WU02:FS01:Upload 57.53%
08:56:39:WU02:FS01:Upload 62.41%
08:56:45:WU02:FS01:Upload 68.57%
08:56:51:WU02:FS01:Upload 80.90%
08:56:57:WU02:FS01:Upload 89.12%
08:57:03:WU02:FS01:Upload 97.85%
08:57:08:WU02:FS01:Upload complete
08:57:08:WU02:FS01:Server responded PLEASE_WAIT (464)
08:57:08:WARNING:WU02:FS01:Failed to send results, will try again later
08:57:09:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
08:57:09:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
08:57:09:WU02:FS01:Connecting to 140.163.4.231:8080
08:57:21:WU02:FS01:Upload 0.51%
08:57:27:WU02:FS01:Upload 7.19%
08:57:33:WU02:FS01:Upload 14.38%
08:57:39:WU02:FS01:Upload 26.71%
08:57:45:WU02:FS01:Upload 29.28%
08:57:51:WU02:FS01:Upload 31.33%
08:57:57:WU02:FS01:Upload 33.13%
08:58:03:WU02:FS01:Upload 42.12%
08:58:09:WU02:FS01:Upload 52.65%
08:58:15:WU02:FS01:Upload 64.98%
08:58:21:WU02:FS01:Upload 75.51%
08:58:27:WU02:FS01:Upload 77.31%
08:58:33:WU02:FS01:Upload 79.88%
08:58:39:WU02:FS01:Upload 87.84%
08:58:45:WU02:FS01:Upload 100.00%
08:58:45:WU02:FS01:Upload complete
08:58:45:WU02:FS01:Server responded PLEASE_WAIT (464)
08:58:45:WARNING:WU02:FS01:Failed to send results, will try again later
08:58:45:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
08:58:45:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
08:58:45:WU02:FS01:Connecting to 140.163.4.231:8080
08:58:51:WU02:FS01:Upload 10.79%
08:58:57:WU02:FS01:Upload 22.86%
08:59:03:WU02:FS01:Upload 25.17%
08:59:09:WU02:FS01:Upload 36.98%
08:59:15:WU02:FS01:Upload 45.72%
08:59:21:WU02:FS01:Upload 57.79%
08:59:27:WU02:FS01:Upload 69.34%
08:59:33:WU02:FS01:Upload 76.79%
08:59:39:WU02:FS01:Upload 78.33%
08:59:45:WU02:FS01:Upload 82.19%
08:59:51:WU02:FS01:Upload 91.18%
08:59:55:WU02:FS01:Upload complete
08:59:55:WU02:FS01:Server responded PLEASE_WAIT (464)
08:59:55:WARNING:WU02:FS01:Failed to send results, will try again later
09:00:22:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:00:22:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:00:22:WU02:FS01:Connecting to 140.163.4.231:8080
09:00:28:WU02:FS01:Upload 2.05%
09:00:34:WU02:FS01:Upload 5.65%
09:00:40:WU02:FS01:Upload 15.67%
09:00:46:WU02:FS01:Upload 27.22%
09:00:58:WU02:FS01:Upload 49.57%
09:01:04:WU02:FS01:Upload 53.93%
09:01:10:WU02:FS01:Upload 55.99%
09:01:16:WU02:FS01:Upload 57.79%
09:01:22:WU02:FS01:Upload 66.78%
09:01:28:WU02:FS01:Upload 78.85%
09:01:34:WU02:FS01:Upload 87.84%
09:01:40:WU02:FS01:Upload 95.29%
09:01:46:WU02:FS01:Upload 99.14%
09:01:50:WU02:FS01:Upload complete
09:01:50:WU02:FS01:Server responded PLEASE_WAIT (464)
09:01:50:WARNING:WU02:FS01:Failed to send results, will try again later
09:02:59:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:02:59:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:02:59:WU02:FS01:Connecting to 140.163.4.231:8080
09:03:05:WU02:FS01:Upload 1.80%
09:03:11:WU02:FS01:Upload 4.37%
09:03:17:WU02:FS01:Upload 13.10%
09:03:23:WU02:FS01:Upload 25.17%
09:03:29:WU02:FS01:Upload 38.01%
09:03:35:WU02:FS01:Upload 43.40%
09:03:41:WU02:FS01:Upload 47.26%
09:03:47:WU02:FS01:Upload 49.83%
09:03:53:WU02:FS01:Upload 53.93%
09:03:59:WU02:FS01:Upload 65.49%
09:04:05:WU02:FS01:Upload 71.91%
09:04:11:WU02:FS01:Upload 82.19%
09:04:17:WU02:FS01:Upload 92.97%
09:04:23:WU02:FS01:Upload 95.80%
09:04:29:WU02:FS01:Upload 98.88%
09:04:31:WU02:FS01:Upload complete
09:04:31:WU02:FS01:Server responded PLEASE_WAIT (464)
09:04:31:WARNING:WU02:FS01:Failed to send results, will try again later
09:07:14:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:07:14:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:07:14:WU02:FS01:Connecting to 140.163.4.231:8080
09:07:20:WU02:FS01:Upload 3.60%
09:07:26:WU02:FS01:Upload 12.33%
09:07:32:WU02:FS01:Upload 23.63%
09:07:38:WU02:FS01:Upload 35.70%
09:07:44:WU02:FS01:Upload 41.35%
09:07:50:WU02:FS01:Upload 50.85%
09:07:56:WU02:FS01:Upload 52.39%
09:08:02:WU02:FS01:Upload 55.73%
09:08:08:WU02:FS01:Upload 64.98%
09:08:14:WU02:FS01:Upload 76.54%
09:08:20:WU02:FS01:Upload 88.86%
09:08:25:WU02:FS01:Upload complete
09:08:25:WU02:FS01:Server responded PLEASE_WAIT (464)
09:08:25:WARNING:WU02:FS01:Failed to send results, will try again later
09:14:05:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:14:05:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:14:05:WU02:FS01:Connecting to 140.163.4.231:8080
09:14:11:WU02:FS01:Upload 0.51%
09:14:17:WU02:FS01:Upload 2.83%
09:14:23:WU02:FS01:Upload 10.02%
09:14:29:WU02:FS01:Upload 22.60%
09:14:35:WU02:FS01:Upload 34.93%
09:14:41:WU02:FS01:Upload 38.27%
09:14:48:WU02:FS01:Upload 40.84%
09:14:54:WU02:FS01:Upload 47.77%
09:15:08:WU02:FS01:Upload 55.48%
09:15:15:WU02:FS01:Upload 58.04%
09:15:21:WU02:FS01:Upload 60.36%
09:15:27:WU02:FS01:Upload 67.55%
09:15:33:WU02:FS01:Upload 78.59%
09:15:39:WU02:FS01:Upload 88.09%
09:15:45:WU02:FS01:Upload 98.11%
09:15:51:WU02:FS01:Upload 99.14%
09:15:54:WU02:FS01:Upload complete
09:15:54:WU02:FS01:Server responded PLEASE_WAIT (464)
09:15:54:WARNING:WU02:FS01:Failed to send results, will try again later
09:25:10:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:25:10:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:25:10:WU02:FS01:Connecting to 140.163.4.231:8080
09:25:16:WU02:FS01:Upload 4.62%
09:25:22:WU02:FS01:Upload 14.38%
09:25:28:WU02:FS01:Upload 27.22%
09:25:34:WU02:FS01:Upload 38.27%
09:25:40:WU02:FS01:Upload 49.57%
09:25:46:WU02:FS01:Upload 51.11%
09:25:52:WU02:FS01:Upload 54.19%
09:25:58:WU02:FS01:Upload 56.25%
09:26:07:WU02:FS01:Upload 63.44%
09:26:13:WU02:FS01:Upload 71.66%
09:26:19:WU02:FS01:Upload 81.93%
09:26:25:WU02:FS01:Upload 94.77%
09:26:27:WU02:FS01:Upload complete
09:26:27:WU02:FS01:Server responded PLEASE_WAIT (464)
09:26:27:WARNING:WU02:FS01:Failed to send results, will try again later
09:43:07:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
09:43:07:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
09:43:07:WU02:FS01:Connecting to 140.163.4.231:8080
09:43:13:WU02:FS01:Upload 2.83%
09:43:19:WU02:FS01:Upload 5.91%
09:43:25:WU02:FS01:Upload 10.53%
09:43:31:WU02:FS01:Upload 17.72%
09:43:37:WU02:FS01:Upload 30.31%
09:43:43:WU02:FS01:Upload 39.04%
09:43:49:WU02:FS01:Upload 42.38%
09:44:01:WU02:FS01:Upload 43.40%
09:44:07:WU02:FS01:Upload 51.88%
09:44:15:WU02:FS01:Upload 54.96%
09:44:21:WU02:FS01:Upload 59.84%
09:44:27:WU02:FS01:Upload 67.80%
09:44:34:WU02:FS01:Upload 71.40%
09:44:40:WU02:FS01:Upload 74.48%
09:44:53:WU02:FS01:Upload 77.05%
09:45:00:WU02:FS01:Upload 81.16%
09:45:06:WU02:FS01:Upload 86.55%
09:45:12:WU02:FS01:Upload 89.63%
09:45:19:WU02:FS01:Upload 90.41%
09:45:26:WU02:FS01:Upload 96.31%
09:45:31:WU02:FS01:Upload complete
09:45:31:WU02:FS01:Server responded PLEASE_WAIT (464)
09:45:31:WARNING:WU02:FS01:Failed to send results, will try again later
10:12:09:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
10:12:09:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
10:12:09:WU02:FS01:Connecting to 140.163.4.231:8080
10:12:21:WU02:FS01:Upload 0.51%
10:12:27:WU02:FS01:Upload 3.08%
10:12:33:WU02:FS01:Upload 6.68%
10:12:39:WU02:FS01:Upload 11.81%
10:12:46:WU02:FS01:Upload 22.86%
10:12:52:WU02:FS01:Upload 32.10%
10:13:02:WU02:FS01:Upload 41.09%
10:13:08:WU02:FS01:Upload 42.89%
10:13:14:WU02:FS01:Upload 45.97%
10:13:20:WU02:FS01:Upload 55.22%
10:13:26:WU02:FS01:Upload 64.47%
10:13:32:WU02:FS01:Upload 74.74%
10:13:38:WU02:FS01:Upload 77.56%
10:13:44:WU02:FS01:Upload 81.42%
10:13:50:WU02:FS01:Upload 82.19%
10:13:57:WU02:FS01:Upload 86.30%
10:14:05:WU02:FS01:Upload 97.34%
10:14:10:WU02:FS01:Upload complete
10:14:10:WU02:FS01:Server responded PLEASE_WAIT (464)
10:14:10:WARNING:WU02:FS01:Failed to send results, will try again later
10:59:08:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:11752 run:0 clone:10571 gen:3 core:0x22 unit:0x000000088ca304e75e6d6dd447d35666
10:59:08:WU02:FS01:Uploading 24.33MiB to 140.163.4.231
10:59:08:WU02:FS01:Connecting to 140.163.4.231:8080
10:59:14:WU02:FS01:Upload 0.51%
10:59:20:WU02:FS01:Upload 3.34%
10:59:26:WU02:FS01:Upload 5.65%
10:59:32:WU02:FS01:Upload 14.38%
10:59:38:WU02:FS01:Upload 27.22%
10:59:46:WU02:FS01:Upload 33.39%
10:59:53:WU02:FS01:Upload 36.98%
10:59:59:WU02:FS01:Upload 39.81%
11:00:05:WU02:FS01:Upload 41.61%
11:00:11:WU02:FS01:Upload 48.03%
11:00:17:WU02:FS01:Upload 60.61%
11:00:23:WU02:FS01:Upload 73.45%
11:00:30:WU02:FS01:Upload 84.24%
11:00:36:WU02:FS01:Upload 92.20%
11:00:42:WU02:FS01:Upload 93.49%
11:00:48:WU02:FS01:Upload 96.57%
11:00:50:WU02:FS01:Upload complete
11:00:50:WU02:FS01:Server responded PLEASE_WAIT (464)
11:00:50:WARNING:WU02:FS01:Failed to send results, will try again later
Re: Can't upload to 140.163.4.231 again
Posted: Fri Apr 17, 2020 12:23 pm
by Neil-B
If the Expiration Date has past the client will delete the WU itself … Until such time there is potentially value for the science in the WU and so letting your client keep trying to resend it until such time as the server can fully receive it is the best course of action … There may be other factors in play so someone more technical than will hopefully review your logs and comment accordingly.
Re: Can't upload to 140.163.4.231 again
Posted: Fri Apr 17, 2020 1:32 pm
by NathanJanssens
Thanks for that, I'll just let it run then.
One of two was just able to finally upload the results!
Code: Select all
13:26:36:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:11749 run:0 clone:3565 gen:24 core:0x22 unit:0x0000002d8ca304e75e6a800f3e735ca5
13:26:36:WU01:FS02:Uploading 12.57MiB to 140.163.4.231
13:26:36:WU01:FS02:Connecting to 140.163.4.231:8080
13:27:30:WU01:FS02:Upload 0.99%
13:27:36:WU01:FS02:Upload 5.47%
13:27:42:WU01:FS02:Upload 10.44%
13:27:48:WU01:FS02:Upload 26.36%
13:27:54:WU01:FS02:Upload 44.76%
13:28:01:WU01:FS02:Upload 54.71%
13:28:07:WU01:FS02:Upload 69.63%
13:28:14:WU01:FS02:Upload 81.57%
13:28:20:WU01:FS02:Upload 84.55%
13:28:26:WU01:FS02:Upload 98.48%
13:28:31:WU01:FS02:Upload complete
13:28:31:WU01:FS02:Server responded WORK_ACK (400)
13:28:31:WU01:FS02:Final credit estimate, 14598.00 points
13:28:31:WU01:FS02:Cleaning up
Re: Can't upload to 140.163.4.231 again
Posted: Fri Apr 17, 2020 2:41 pm
by Neil-B
YAY
… Fingers crossed for the next !!