Failed Transfer 128.252.203.2

[ponley]

Looking through the threads I see that this problem has been reported on a couple different servers in the last few months.
I have 8 27MB WUs stacked up here that the client has tried over 20 times each to upload getting 1/4 to 1/2 done before failing.

Code: Select all

05:10:31:WU03:FS00:Upload 6.05%
05:10:37:WU08:FS00:Upload 2.69%
05:10:37:WU03:FS00:Upload 6.50%
05:10:43:WU08:FS00:Upload 2.92%
05:10:45:WU01:FS00:Upload 47.78%
05:10:45:WARNING:WU01:FS00:Exception: Failed to send results to work server: Transfer failed
05:10:45:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:11626 run:1 clone:0 gen:10 core:0xa4 unit:0x0000000d4c71bbb056971e4d613d45b0
05:10:46:WU01:FS00:Uploading 27.86MiB to 128.252.203.2
05:10:46:WU01:FS00:Connecting to 128.252.203.2:8080
05:10:47:WU03:FS00:Upload 6.95%
05:10:54:WU08:FS00:Upload 3.36%
05:10:56:WU03:FS00:Upload 7.17%
05:10:56:WU01:FS00:Upload 0.22%
05:11:00:WU08:FS00:Upload 3.59%
05:11:07:WU08:FS00:Upload 3.81%
05:11:07:WU01:FS00:Upload 0.45%
05:11:14:WU03:FS00:Upload 7.40%
05:11:15:WU08:FS00:Upload 4.26%
05:11:18:WU01:FS00:Upload 0.90%
05:11:24:WU08:FS00:Upload 4.71%
05:11:24:WU01:FS00:Upload 1.12%
05:11:28:WU03:FS00:Upload 7.62%
05:11:32:WU01:FS00:Upload 1.35%
05:11:33:WU08:FS00:Upload 5.16%
05:11:38:WU01:FS00:Upload 1.57%
05:11:43:WU08:FS00:Upload 5.61%
05:11:51:WU01:FS00:Upload 2.02%
05:11:54:WU08:FS00:Upload 6.06%
05:11:55:WU03:FS00:Upload 7.84%
05:11:58:WU01:FS00:Upload 2.24%

[bruce]

How long did the upload process work on individual WUs? You didn't post enough of the log for me to figure that out. Each server times out if an upload takes longer than N minutes.

[ponley]

I suspected that was at play here but could not find that little gem of information anywhere on the forum. Thre are multiple issues going on here. The situation had snowballed into 8 WUs trying to upload and failing simultaneously as you can see from this excerpt showing WU01 failing midway through WU00. I only have a 256K uplink so 27MiB is going to take some time under the best of conditions and once one fails if the client doesn't look to see if it's already uploading a WU before starting another upload then anytime there is a failed upload you risk this sort of situation on slow uplinks. This can be clearly seen as the upload of WU01 stops for 17 minutes when the upload of WU00 starts. Further more when FAH attempts to upload a WU it saturates my uplink resulting in latency times of 1000-2000 ms which means this problem has been breaking all kinds of other processes on timeouts for the last 3 days as well. If the server is going to time out the connections the client should not be starting a new upload until it finishes an existing upload.

Code: Select all

21:05:05:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:11626 run:1 clone:0 gen:10 core:0xa4 unit:0x0000000d4c71bbb056971e4d613d45b0
21:05:05:WU01:FS00:Uploading 27.86MiB to 128.252.203.2
21:05:05:WU01:FS00:Connecting to 128.252.203.2:8080
21:05:07:WU07:FS00:Upload 63.97%
21:05:13:WU01:FS00:Upload 0.45%
21:05:17:WU07:FS00:Upload 64.20%
21:05:21:WU01:FS00:Upload 1.12%
21:05:23:WU07:FS00:Upload 64.42%
21:05:30:WU07:FS00:Upload 64.65%
21:05:30:WU01:FS00:Upload 1.57%
21:05:36:WU07:FS00:Upload 64.87%
21:05:38:WU01:FS00:Upload 2.02%
21:05:45:WU07:FS00:Upload 65.10%
21:05:47:WU01:FS00:Upload 2.47%
21:05:56:WU07:FS00:Upload 65.55%
21:05:58:WU01:FS00:Upload 2.92%
21:06:02:WU07:FS00:Upload 65.77%
21:06:08:WU01:FS00:Upload 3.36%
21:06:11:WU07:FS00:Upload 66.22%
21:06:15:WU01:FS00:Upload 3.59%
21:06:19:WU07:FS00:Upload 66.67%
21:06:26:WU01:FS00:Upload 3.81%
21:06:26:WU07:FS00:Upload 67.12%
21:06:34:WU01:FS00:Upload 4.26%
21:06:40:WU07:FS00:Upload 67.57%
21:06:42:WU01:FS00:Upload 4.71%
21:06:49:WU01:FS00:Upload 5.16%
21:06:52:WU07:FS00:Upload 68.01%
21:06:58:WU01:FS00:Upload 5.61%
21:06:59:WU07:FS00:Upload 68.24%
21:07:05:WU07:FS00:Upload 68.46%
21:07:07:WU01:FS00:Upload 6.06%
21:07:14:WU01:FS00:Upload 6.51%
21:07:14:WU07:FS00:Upload 68.69%
21:07:20:WU07:FS00:Upload 68.91%
21:07:23:WU01:FS00:Upload 6.95%
******************************* Date: 2016-03-15 *******************************
21:07:29:WU07:FS00:Upload 69.36%
21:07:35:WU01:FS00:Upload 7.40%
21:07:35:WU07:FS00:Upload 69.59%
21:07:43:WU01:FS00:Upload 7.63%
21:07:51:WU01:FS00:Upload 8.08%
21:07:58:WU01:FS00:Upload 8.75%
21:08:04:WU01:FS00:Upload 9.20%
21:08:10:WU01:FS00:Upload 9.65%
21:08:18:WU01:FS00:Upload 10.32%
21:08:25:WU01:FS00:Upload 10.99%
21:08:32:WU01:FS00:Upload 11.67%
21:08:39:WU01:FS00:Upload 12.11%
21:08:45:WU01:FS00:Upload 12.56%
21:08:53:WU01:FS00:Upload 13.01%
21:08:59:WU01:FS00:Upload 13.46%
21:09:08:WU01:FS00:Upload 14.13%
21:09:15:WU01:FS00:Upload 14.81%
21:09:22:WU01:FS00:Upload 15.48%
21:09:28:WU01:FS00:Upload 16.15%
21:09:35:WU01:FS00:Upload 16.82%
21:09:42:WU01:FS00:Upload 17.50%
21:09:49:WU01:FS00:Upload 18.17%
21:09:55:WU01:FS00:Upload 18.84%
21:10:02:WU01:FS00:Upload 19.52%
21:10:08:WU01:FS00:Upload 20.19%
21:10:15:WU01:FS00:Upload 20.86%
21:10:22:WU01:FS00:Upload 21.54%
21:10:29:WU01:FS00:Upload 22.21%
21:10:35:WU01:FS00:Upload 22.88%
21:10:41:WU01:FS00:Upload 23.11%
21:10:47:WU01:FS00:Upload 23.33%
21:10:54:WU01:FS00:Upload 23.78%
21:11:00:WU01:FS00:Upload 24.45%
21:11:07:WU01:FS00:Upload 25.12%
21:11:13:WU01:FS00:Upload 25.80%
21:11:20:WU01:FS00:Upload 26.47%
21:11:28:WU01:FS00:Upload 27.14%
21:11:34:WU01:FS00:Upload 27.82%
21:11:40:WU01:FS00:Upload 28.49%
21:11:46:WU01:FS00:Upload 29.16%
21:11:54:WU01:FS00:Upload 29.84%
21:12:00:WU01:FS00:Upload 30.28%
21:12:08:WU01:FS00:Upload 31.18%
21:12:15:WU01:FS00:Upload 31.85%
21:12:21:WU01:FS00:Upload 32.30%
21:12:27:WU01:FS00:Upload 32.98%
21:12:33:WU01:FS00:Upload 33.43%
21:12:40:WU01:FS00:Upload 34.10%
21:12:47:WU01:FS00:Upload 34.77%
21:12:55:WU01:FS00:Upload 35.67%
21:13:03:WU01:FS00:Upload 36.34%
21:13:12:WU01:FS00:Upload 37.24%
21:13:19:WU01:FS00:Upload 37.91%
21:13:26:WU01:FS00:Upload 38.58%
21:13:32:WU01:FS00:Upload 39.26%
21:13:38:WU01:FS00:Upload 39.71%
21:13:44:WU01:FS00:Upload 40.38%
21:13:50:WU01:FS00:Upload 40.83%
21:13:58:WU01:FS00:Upload 41.50%
21:14:05:WU01:FS00:Upload 42.17%
21:14:12:WU01:FS00:Upload 42.85%
21:14:19:WU01:FS00:Upload 43.52%
21:14:26:WU01:FS00:Upload 44.19%
21:14:33:WU01:FS00:Upload 44.87%
21:14:40:WU01:FS00:Upload 45.54%
21:14:47:WU01:FS00:Upload 46.21%
21:14:55:WU01:FS00:Upload 46.88%
21:15:01:WU01:FS00:Upload 47.33%
21:15:09:WU01:FS00:Upload 48.01%
21:15:15:WU01:FS00:Upload 48.68%
21:15:22:WU01:FS00:Upload 49.35%
21:15:28:WU01:FS00:Upload 50.03%
21:15:36:WU01:FS00:Upload 50.70%
21:15:44:WU01:FS00:Upload 51.37%
21:15:51:WU01:FS00:Upload 52.04%
21:15:57:WU01:FS00:Upload 52.72%
21:16:04:WU01:FS00:Upload 53.39%
21:16:11:WU01:FS00:Upload 54.06%
21:16:17:WU01:FS00:Upload 54.74%
21:16:24:WU01:FS00:Upload 55.41%
21:16:32:WU01:FS00:Upload 56.31%
21:16:40:WU01:FS00:Upload 56.98%
21:16:46:WU01:FS00:Upload 57.65%
21:16:53:WU01:FS00:Upload 58.33%
21:17:00:WU01:FS00:Upload 59.00%
21:17:06:WU01:FS00:Upload 59.67%
21:17:12:WU01:FS00:Upload 60.34%
21:17:22:WU01:FS00:Upload 61.02%
21:17:29:WU01:FS00:Upload 61.69%
21:17:35:WU01:FS00:Upload 62.36%
21:17:41:WU01:FS00:Upload 63.04%
21:17:48:WU01:FS00:Upload 63.71%
21:17:55:WU01:FS00:Upload 64.38%
21:18:01:WU01:FS00:Upload 65.06%
21:18:08:WU01:FS00:Upload 65.73%
21:18:15:WU01:FS00:Upload 66.40%
21:18:21:WU01:FS00:Upload 67.07%
21:18:27:WU01:FS00:Upload 67.52%
21:18:35:WU01:FS00:Upload 68.42%
21:18:42:WU01:FS00:Upload 69.09%
21:18:48:WU01:FS00:Upload 69.54%
21:18:55:WU01:FS00:Upload 70.22%
21:19:01:WU01:FS00:Upload 70.66%
21:19:07:WU01:FS00:Upload 71.34%
21:19:13:WU01:FS00:Upload 72.01%
21:19:20:WU01:FS00:Upload 72.68%
21:19:26:WU01:FS00:Upload 73.36%
21:19:33:WU01:FS00:Upload 74.03%
21:19:39:WU01:FS00:Upload 74.70%
21:19:46:WU01:FS00:Upload 75.37%
21:19:52:WU01:FS00:Upload 75.82%
21:19:58:WU01:FS00:Upload 76.27%
21:20:05:WU01:FS00:Upload 76.95%
21:20:14:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:11630 run:0 clone:194 gen:15 core:0xa4 unit:0x000000124c71bbb0569e80e91f6ce517
21:20:14:WU00:FS00:Uploading 27.85MiB to 128.252.203.2
21:20:14:WU00:FS00:Connecting to 128.252.203.2:8080
21:20:22:WU00:FS00:Upload 0.90%
21:20:30:WU00:FS00:Upload 1.57%
21:20:39:WU00:FS00:Upload 2.24%
21:20:46:WU00:FS00:Upload 2.92%
21:20:53:WU00:FS00:Upload 3.37%
21:21:00:WU00:FS00:Upload 4.04%
21:21:08:WU00:FS00:Upload 4.71%
21:21:16:WU00:FS00:Upload 5.39%
21:21:23:WU00:FS00:Upload 6.06%
21:21:31:WU00:FS00:Upload 6.73%
21:21:38:WU00:FS00:Upload 7.41%
21:21:46:WU00:FS00:Upload 8.08%
21:21:52:WU00:FS00:Upload 8.75%
21:21:58:WU00:FS00:Upload 9.20%
21:22:05:WU00:FS00:Upload 9.88%
21:22:12:WU00:FS00:Upload 10.32%
21:22:18:WU00:FS00:Upload 10.77%
21:22:26:WU00:FS00:Upload 11.45%
21:22:33:WU00:FS00:Upload 12.12%
21:22:41:WU00:FS00:Upload 12.79%
21:22:49:WU00:FS00:Upload 13.47%
21:22:56:WU00:FS00:Upload 14.14%
21:23:02:WU00:FS00:Upload 14.59%
21:23:09:WU00:FS00:Upload 15.26%
21:23:17:WU00:FS00:Upload 15.93%
21:23:24:WU00:FS00:Upload 16.61%
21:23:29:WU07:FS00:Upload 70.03%
[93m21:23:29:WARNING:WU07:FS00:Exception: Failed to send results to work server: Transfer failed[0m
21:23:31:WU00:FS00:Upload 17.06%
21:23:37:WU00:FS00:Upload 17.51%
21:23:43:WU00:FS00:Upload 18.18%
21:23:49:WU00:FS00:Upload 18.63%
21:23:58:WU00:FS00:Upload 19.30%
21:24:04:WU00:FS00:Upload 19.75%
21:24:11:WU00:FS00:Upload 20.42%
21:24:19:WU00:FS00:Upload 21.10%
21:24:26:WU00:FS00:Upload 21.77%
21:24:32:WU00:FS00:Upload 22.22%
21:24:39:WU00:FS00:Upload 22.89%
21:24:46:WU00:FS00:Upload 23.57%
21:24:52:WU00:FS00:Upload 24.01%
21:24:59:WU00:FS00:Upload 24.69%
21:25:05:WU00:FS00:Upload 25.14%
21:25:13:WU00:FS00:Upload 25.81%
21:25:21:WU00:FS00:Upload 26.48%
21:25:27:WU00:FS00:Upload 26.93%
21:25:34:WU00:FS00:Upload 27.61%
21:25:40:WU00:FS00:Upload 28.05%
21:25:47:WU00:FS00:Upload 28.73%
21:25:53:WU00:FS00:Upload 29.18%
21:26:00:WU00:FS00:Upload 29.85%
21:26:06:WU00:FS00:Upload 30.30%
21:26:13:WU00:FS00:Upload 30.97%
21:26:20:WU00:FS00:Upload 31.65%
21:26:26:WU00:FS00:Upload 32.09%
21:26:33:WU00:FS00:Upload 32.77%
21:26:39:WU00:FS00:Upload 33.22%
21:26:45:WU00:FS00:Upload 33.89%
21:26:51:WU00:FS00:Upload 34.34%
21:26:58:WU00:FS00:Upload 35.01%
21:27:04:WU00:FS00:Upload 35.46%
21:27:11:WU00:FS00:Upload 36.13%
21:27:17:WU00:FS00:Upload 36.58%
21:27:24:WU00:FS00:Upload 37.26%
21:27:32:WU00:FS00:Upload 37.93%
21:27:39:WU00:FS00:Upload 38.60%
21:27:45:WU00:FS00:Upload 39.05%
21:27:51:WU00:FS00:Upload 39.72%
21:27:57:WU00:FS00:Upload 40.17%
21:28:05:WU00:FS00:Upload 40.85%
21:28:11:WU00:FS00:Upload 41.52%
21:28:17:WU00:FS00:Upload 41.97%
21:28:23:WU00:FS00:Upload 42.64%
21:28:29:WU00:FS00:Upload 43.09%
21:28:36:WU00:FS00:Upload 43.76%
21:28:42:WU00:FS00:Upload 44.21%
21:28:49:WU00:FS00:Upload 44.89%
21:28:57:WU00:FS00:Upload 45.56%
21:29:04:WU00:FS00:Upload 46.23%
21:29:10:WU00:FS00:Upload 46.68%
21:29:17:WU00:FS00:Upload 47.36%
21:29:23:WU00:FS00:Upload 47.80%
21:29:29:WU00:FS00:Upload 48.25%
21:29:35:WU00:FS00:Upload 48.70%
21:29:42:WU00:FS00:Upload 49.15%
21:29:50:WU00:FS00:Upload 49.82%
21:29:58:WU00:FS00:Upload 50.50%
21:30:06:WU00:FS00:Upload 50.95%
21:30:13:WU00:FS00:Upload 51.62%
21:30:21:WU00:FS00:Upload 52.29%
21:30:28:WU00:FS00:Upload 52.97%
21:30:34:WU00:FS00:Upload 53.42%
21:30:40:WU00:FS00:Upload 54.09%
21:30:47:WU00:FS00:Upload 54.54%
21:30:53:WU00:FS00:Upload 55.21%
21:31:00:WU00:FS00:Upload 55.66%
21:31:06:WU00:FS00:Upload 56.33%
21:31:13:WU00:FS00:Upload 56.78%
21:31:19:WU00:FS00:Upload 57.46%
21:31:25:WU00:FS00:Upload 57.90%
21:31:32:WU00:FS00:Upload 58.58%
21:31:38:WU00:FS00:Upload 59.03%
21:31:46:WU00:FS00:Upload 59.70%
21:31:53:WU00:FS00:Upload 60.37%
21:32:00:WU00:FS00:Upload 61.05%
21:32:07:WU00:FS00:Upload 61.72%
21:32:13:WU00:FS00:Upload 62.17%
21:32:20:WU00:FS00:Upload 62.84%
21:32:26:WU00:FS00:Upload 63.29%
21:32:33:WU00:FS00:Upload 63.74%
21:32:39:WU00:FS00:Upload 64.41%
21:32:45:WU00:FS00:Upload 64.86%
21:32:52:WU00:FS00:Upload 65.53%
21:33:00:WU00:FS00:Upload 66.21%
21:33:06:WU00:FS00:Upload 66.88%
21:33:12:WU00:FS00:Upload 67.33%
21:33:18:WU00:FS00:Upload 68.00%
21:33:25:WU00:FS00:Upload 68.45%
21:33:33:WU00:FS00:Upload 69.13%
21:33:41:WU00:FS00:Upload 69.80%
21:33:47:WU00:FS00:Upload 70.25%
21:33:53:WU00:FS00:Upload 70.92%
21:33:59:WU00:FS00:Upload 71.37%
21:34:05:WU00:FS00:Upload 72.04%
21:34:12:WU00:FS00:Upload 72.49%
21:34:18:WU00:FS00:Upload 73.17%
21:34:25:WU00:FS00:Upload 73.61%
21:34:32:WU00:FS00:Upload 74.29%
21:34:38:WU00:FS00:Upload 74.74%
21:34:44:WU00:FS00:Upload 75.41%
21:34:50:WU00:FS00:Upload 75.86%
21:34:58:WU00:FS00:Upload 76.53%
21:35:08:WU00:FS00:Upload 77.21%
21:37:21:WU01:FS00:Upload 77.17%
[93m21:37:21:WARNING:WU01:FS00:Exception: Failed to send results to work server: Transfer failed[0m
21:51:08:WU00:FS00:Upload 77.88%
[93m21:51:08:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed[0m
[93m01:19:52:WARNING:WU06:FS00:Past final deadline 2016-03-16T01:19:51Z, dumping[0m
01:19:52:WU06:FS00:Cleaning up
02:00:23:Lost lifeline PID 14255, exiting
02:00:24:Clean exit

[bruce]

Have you altered the value of max-packet-size? The default is supposed to be 10 which should prevent you from getting WUs that are 27MIB.

I'll check with the owner of project:11626 and confirm the server is set properly for that project.

It looks like if two WUs are uploading concurrently, each will successfully upload about 70% of 28MiB or about 19 MiB each. If only one WU is uploading, it should get there before the server times out.

[ponley]

I used the recommendations I was given when I set v7 up over 2 years ago. This machine is setup for SMP folding and has worked fine for all that time. The only thing that has changed is that I moved about 4 months ago and am now too far from the CO to get better than 256k uplink speed. Things have been working fine until about 2 weeks ago. As I mentioned the client should not allow a second upload to start until the first is finished. If an upload broke due to a power failure then the client can keep folding WUs while concurrent uploads keep them piling up. This wouldn't be a problem on a 1Mb uplink but presents a problem on lesser connections.

Code: Select all

*********************** Log Started 2016-03-13T21:06:52Z ***********************
21:06:52:************************* Folding@home Client *************************
21:06:52:    Website: http://folding.stanford.edu/
21:06:52:  Copyright: (c) 2009-2014 Stanford University
21:06:52:     Author: Joseph Coffland <[email protected]>
21:06:52:       Args: --child --lifeline 14255 /etc/fahclient/config.xml --run-as
21:06:52:             fahclient --pid-file=/var/run/fahclient.pid --daemon
21:06:52:     Config: /etc/fahclient/config.xml
21:06:52:******************************** Build ********************************
21:06:52:    Version: 7.4.4
21:06:52:       Date: Mar 4 2014
21:06:52:       Time: 12:02:38
21:06:52:    SVN Rev: 4130
21:06:52:     Branch: fah/trunk/client
21:06:52:   Compiler: GNU 4.4.7
21:06:52:    Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
21:06:52:             -fno-unsafe-math-optimizations -msse2
21:06:52:   Platform: linux2 3.2.0-1-amd64
21:06:52:       Bits: 64
21:06:52:       Mode: Release
21:06:52:******************************* System ********************************
21:06:52:        CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz
21:06:52:     CPU ID: GenuineIntel Family 6 Model 23 Stepping 10
21:06:52:       CPUs: 4
21:06:52:     Memory: 7.69GiB
21:06:52:Free Memory: 236.01MiB
21:06:52:    Threads: POSIX_THREADS
21:06:52: OS Version: 3.2
21:06:52:Has Battery: false
21:06:52: On Battery: false
21:06:52: UTC Offset: -5
21:06:52:        PID: 14257
21:06:52:        CWD: /var/lib/fahclient
21:06:52:         OS: Linux 3.2.0-4-amd64 x86_64
21:06:52:    OS Arch: AMD64
21:06:52:       GPUs: 0
21:06:52:       CUDA: Not detected
21:06:52:***********************************************************************
21:06:52:<config>
21:06:52:  <!-- Folding Slot Configuration -->
21:06:52:  <gpu v='false'/>
21:06:52:
21:06:52:  <!-- Slot Control -->
21:06:52:  <power v='FULL'/>
21:06:52:
21:06:52:  <!-- User Information -->
21:06:52:  <passkey v='********************************'/>
21:06:52:  <team v='*****'/>
21:06:52:  <user v='******'/>
21:06:52:
21:06:52:  <!-- Folding Slots -->
21:06:52:  <slot id='0' type='CPU'/>
21:06:52:</config>
21:06:52:Switching to user fahclient
21:06:52:Trying to access database...
21:06:57:Successfully acquired database lock
21:06:57:Enabled folding slot 00: READY cpu:4
21:06:57:WU06:FS00:Sending unit results: id:06 state:SEND error:NO_ERROR project:11625 run:0 clone:166 gen:14 core:0xa4 unit:0x000000104c71bbb0569e7d5a937fc07a
21:06:58:WU06:FS00:Uploading 27.86MiB to 128.252.203.2
21:06:58:WU06:FS00:Connecting to 128.252.203.2:8080
21:06:58:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:11626 run:1 clone:0 gen:10 core:0xa4 unit:0x0000000d4c71bbb056971e4d613d45b0
21:06:58:WU01:FS00:Uploading 27.86MiB to 128.252.203.2
21:06:58:WU01:FS00:Connecting to 128.252.203.2:8080
21:06:58:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:11630 run:0 clone:194 gen:15 core:0xa4 unit:0x000000124c71bbb0569e80e91f6ce517
21:06:58:WU00:FS00:Uploading 27.85MiB to 128.252.203.2
21:06:58:WU00:FS00:Connecting to 128.252.203.2:8080
21:06:58:WU03:FS00:Sending unit results: id:03 state:SEND error:NO_ERROR project:11638 run:1 clone:25 gen:13 core:0xa4 unit:0x000000114c71bbb056987d5fc437ae26
21:06:58:WU03:FS00:Uploading 27.89MiB to 128.252.203.2
21:06:58:WU03:FS00:Connecting to 128.252.203.2:8080
21:06:58:WU07:FS00:Sending unit results: id:07 state:SEND error:NO_ERROR project:11622 run:0 clone:90 gen:26 core:0xa4 unit:0x000000234c71bbb05695ad55d541a60a
21:06:58:WU07:FS00:Uploading 27.84MiB to 128.252.203.2
21:06:58:WU07:FS00:Connecting to 128.252.203.2:8080
21:06:58:WU08:FS00:Sending unit results: id:08 state:SEND error:NO_ERROR project:11639 run:1 clone:47 gen:39 core:0xa4 unit:0x0000002a4c71bbb056987d755b52b38a
21:06:58:WU08:FS00:Uploading 27.86MiB to 128.252.203.2
21:06:58:WU08:FS00:Connecting to 128.252.203.2:8080
21:07:05:WU07:FS00:Upload 0.45%

[bruce]

I contacted the owner of the project and he has made some changes. Your WUs should upload now.