Page 1 of 1
171.67.108.105 / 46
Posted: Sun Oct 09, 2016 10:30 pm
by soa_rru
I take it these are down? as all i'm getting after completing every WU is several failed uploads over a space of say 45mins+ then the dreaded message server didn't like results...dumping
Everything was working fine until today, well i say fine these said servers have been flaky since god knows when but they did always accept my wu's...eventually.
Re: 171.67.108.105 / 46
Posted: Sun Oct 09, 2016 11:04 pm
by Joe_H
I can connect to both servers directly using a browser connection to port 8080, so they are up. If there are problems uploading WU's, please post the sections of your logs showing that as well as the configuration of your system.
Re: 171.67.108.105 / 46
Posted: Sun Oct 09, 2016 11:22 pm
by soa_rru
Yes i can connect directly via my browser as well using 8080, anyway...
Code: Select all
*********************** Log Started 2016-10-09T20:21:01Z ***********************
20:21:01:************************* Folding@home Client *************************
20:21:01: Website: http://folding.stanford.edu/
20:21:01: Copyright: (c) 2009-2014 Stanford University
20:21:01: Author: Joseph Coffland <[email protected]>
20:21:01: Args:
20:21:01: Config: C:/Users/Kris/AppData/Roaming/FAHClient/config.xml
20:21:01:******************************** Build ********************************
20:21:01: Version: 7.4.4
20:21:01: Date: Mar 4 2014
20:21:01: Time: 20:26:54
20:21:01: SVN Rev: 4130
20:21:01: Branch: fah/trunk/client
20:21:01: Compiler: Intel(R) C++ MSVC 1500 mode 1200
20:21:01: Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
20:21:01: /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
20:21:01: Platform: win32 XP
20:21:01: Bits: 32
20:21:01: Mode: Release
20:21:01:******************************* System ********************************
20:21:01: CPU: AMD Phenom(tm) II X6 1055T Processor
20:21:01: CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
20:21:01: CPUs: 6
20:21:01: Memory: 7.99GiB
20:21:01: Free Memory: 6.65GiB
20:21:01: Threads: WINDOWS_THREADS
20:21:01: OS Version: 6.1
20:21:01: Has Battery: false
20:21:01: On Battery: false
20:21:01: UTC Offset: 1
20:21:01: PID: 2732
20:21:01: CWD: C:/Users/Kris/AppData/Roaming/FAHClient
20:21:01: OS: Windows 7
20:21:01: OS Arch: AMD64
20:21:01: GPUs: 2
20:21:01: GPU 0: UNSUPPORTED: NV3 [PCI]
20:21:01: GPU 1: NVIDIA:5 GP104 [GeForce GTX 1070]
20:21:01: CUDA: 6.1
20:21:01: CUDA Driver: 8000
20:21:01:Win32 Service: false
20:21:01:***********************************************************************
20:21:01:<config>
20:21:01: <!-- Folding Core -->
20:21:01: <checkpoint v='3'/>
20:21:01:
20:21:01: <!-- Network -->
20:21:01: <proxy v=':8080'/>
20:21:01:
20:21:01: <!-- Slot Control -->
20:21:01: <power v='full'/>
20:21:01:
20:21:01: <!-- User Information -->
20:21:01: <passkey v='********************************'/>
20:21:01: <team v='98860'/>
20:21:01: <user v='SOA_RRU'/>
20:21:01:
20:21:01: <!-- Folding Slots -->
20:21:01: <slot id='1' type='GPU'/>
20:21:01:</config>
20:21:01:Trying to access database...
20:21:01:Successfully acquired database lock
20:21:01:Enabled folding slot 01: READY gpu:1:GP104 [GeForce GTX 1070]
20:21:01:WU00:FS01:Starting
20:21:01:WU00:FS01:Running FahCore: "E:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Kris/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 2732 -checkpoint 3 -gpu 0 -gpu-vendor nvidia
20:21:01:WU00:FS01:Started FahCore on PID 4424
20:21:02:WU00:FS01:Core PID:4440
20:21:02:WU00:FS01:FahCore 0x21 started
20:21:04:WU00:FS01:0x21:*********************** Log Started 2016-10-09T20:21:04Z ***********************
20:21:04:WU00:FS01:0x21:Project: 9175 (Run 2, Clone 15, Gen 77)
20:21:04:WU00:FS01:0x21:Unit: 0x0000005bab436c6957b24c27c32c33e7
20:21:04:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
20:21:04:WU00:FS01:0x21:Machine: 1
20:21:04:WU00:FS01:0x21:Digital signatures verified
20:21:04:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
20:21:04:WU00:FS01:0x21:Version 0.0.17
20:21:05:WU00:FS01:0x21: Found a checkpoint file
20:21:35:WU00:FS01:0x21:Completed 300000 out of 2500000 steps (12%)
20:21:35:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
20:22:40:WU00:FS01:0x21:Completed 325000 out of 2500000 steps (13%)
20:23:37:WU00:FS01:0x21:Completed 350000 out of 2500000 steps (14%)
20:24:33:WU00:FS01:0x21:Completed 375000 out of 2500000 steps (15%)
20:25:30:WU00:FS01:0x21:Completed 400000 out of 2500000 steps (16%)
20:26:37:WU00:FS01:0x21:Completed 425000 out of 2500000 steps (17%)
20:27:34:WU00:FS01:0x21:Completed 450000 out of 2500000 steps (18%)
20:28:31:WU00:FS01:0x21:Completed 475000 out of 2500000 steps (19%)
20:29:28:WU00:FS01:0x21:Completed 500000 out of 2500000 steps (20%)
20:30:34:WU00:FS01:0x21:Completed 525000 out of 2500000 steps (21%)
20:31:31:WU00:FS01:0x21:Completed 550000 out of 2500000 steps (22%)
20:32:28:WU00:FS01:0x21:Completed 575000 out of 2500000 steps (23%)
20:33:25:WU00:FS01:0x21:Completed 600000 out of 2500000 steps (24%)
20:34:32:WU00:FS01:0x21:Completed 625000 out of 2500000 steps (25%)
20:35:28:WU00:FS01:0x21:Completed 650000 out of 2500000 steps (26%)
20:36:25:WU00:FS01:0x21:Completed 675000 out of 2500000 steps (27%)
20:37:22:WU00:FS01:0x21:Completed 700000 out of 2500000 steps (28%)
20:38:29:WU00:FS01:0x21:Completed 725000 out of 2500000 steps (29%)
20:39:27:WU00:FS01:0x21:Completed 750000 out of 2500000 steps (30%)
20:40:24:WU00:FS01:0x21:Completed 775000 out of 2500000 steps (31%)
20:41:21:WU00:FS01:0x21:Completed 800000 out of 2500000 steps (32%)
20:42:29:WU00:FS01:0x21:Completed 825000 out of 2500000 steps (33%)
20:43:26:WU00:FS01:0x21:Completed 850000 out of 2500000 steps (34%)
20:44:23:WU00:FS01:0x21:Completed 875000 out of 2500000 steps (35%)
20:45:21:WU00:FS01:0x21:Completed 900000 out of 2500000 steps (36%)
20:46:27:WU00:FS01:0x21:Completed 925000 out of 2500000 steps (37%)
20:47:24:WU00:FS01:0x21:Completed 950000 out of 2500000 steps (38%)
20:48:21:WU00:FS01:0x21:Completed 975000 out of 2500000 steps (39%)
20:49:18:WU00:FS01:0x21:Completed 1000000 out of 2500000 steps (40%)
20:50:24:WU00:FS01:0x21:Completed 1025000 out of 2500000 steps (41%)
20:51:21:WU00:FS01:0x21:Completed 1050000 out of 2500000 steps (42%)
20:52:18:WU00:FS01:0x21:Completed 1075000 out of 2500000 steps (43%)
20:53:15:WU00:FS01:0x21:Completed 1100000 out of 2500000 steps (44%)
20:54:23:WU00:FS01:0x21:Completed 1125000 out of 2500000 steps (45%)
20:55:20:WU00:FS01:0x21:Completed 1150000 out of 2500000 steps (46%)
20:56:17:WU00:FS01:0x21:Completed 1175000 out of 2500000 steps (47%)
20:57:14:WU00:FS01:0x21:Completed 1200000 out of 2500000 steps (48%)
20:58:20:WU00:FS01:0x21:Completed 1225000 out of 2500000 steps (49%)
20:59:17:WU00:FS01:0x21:Completed 1250000 out of 2500000 steps (50%)
21:00:14:WU00:FS01:0x21:Completed 1275000 out of 2500000 steps (51%)
21:01:10:WU00:FS01:0x21:Completed 1300000 out of 2500000 steps (52%)
21:02:17:WU00:FS01:0x21:Completed 1325000 out of 2500000 steps (53%)
21:03:14:WU00:FS01:0x21:Completed 1350000 out of 2500000 steps (54%)
21:04:11:WU00:FS01:0x21:Completed 1375000 out of 2500000 steps (55%)
21:05:08:WU00:FS01:0x21:Completed 1400000 out of 2500000 steps (56%)
21:06:14:WU00:FS01:0x21:Completed 1425000 out of 2500000 steps (57%)
21:07:11:WU00:FS01:0x21:Completed 1450000 out of 2500000 steps (58%)
21:08:08:WU00:FS01:0x21:Completed 1475000 out of 2500000 steps (59%)
21:09:05:WU00:FS01:0x21:Completed 1500000 out of 2500000 steps (60%)
21:10:12:WU00:FS01:0x21:Completed 1525000 out of 2500000 steps (61%)
21:11:09:WU00:FS01:0x21:Completed 1550000 out of 2500000 steps (62%)
21:12:05:WU00:FS01:0x21:Completed 1575000 out of 2500000 steps (63%)
21:13:02:WU00:FS01:0x21:Completed 1600000 out of 2500000 steps (64%)
21:14:09:WU00:FS01:0x21:Completed 1625000 out of 2500000 steps (65%)
21:15:06:WU00:FS01:0x21:Completed 1650000 out of 2500000 steps (66%)
21:16:03:WU00:FS01:0x21:Completed 1675000 out of 2500000 steps (67%)
21:16:59:WU00:FS01:0x21:Completed 1700000 out of 2500000 steps (68%)
21:18:06:WU00:FS01:0x21:Completed 1725000 out of 2500000 steps (69%)
21:19:03:WU00:FS01:0x21:Completed 1750000 out of 2500000 steps (70%)
21:20:00:WU00:FS01:0x21:Completed 1775000 out of 2500000 steps (71%)
21:20:57:WU00:FS01:0x21:Completed 1800000 out of 2500000 steps (72%)
21:22:03:WU00:FS01:0x21:Completed 1825000 out of 2500000 steps (73%)
21:23:00:WU00:FS01:0x21:Completed 1850000 out of 2500000 steps (74%)
21:23:57:WU00:FS01:0x21:Completed 1875000 out of 2500000 steps (75%)
21:24:53:WU00:FS01:0x21:Completed 1900000 out of 2500000 steps (76%)
21:26:00:WU00:FS01:0x21:Completed 1925000 out of 2500000 steps (77%)
21:26:57:WU00:FS01:0x21:Completed 1950000 out of 2500000 steps (78%)
21:27:53:WU00:FS01:0x21:Completed 1975000 out of 2500000 steps (79%)
21:28:50:WU00:FS01:0x21:Completed 2000000 out of 2500000 steps (80%)
21:29:57:WU00:FS01:0x21:Completed 2025000 out of 2500000 steps (81%)
21:30:54:WU00:FS01:0x21:Completed 2050000 out of 2500000 steps (82%)
21:31:51:WU00:FS01:0x21:Completed 2075000 out of 2500000 steps (83%)
21:32:48:WU00:FS01:0x21:Completed 2100000 out of 2500000 steps (84%)
21:33:55:WU00:FS01:0x21:Completed 2125000 out of 2500000 steps (85%)
21:34:51:WU00:FS01:0x21:Completed 2150000 out of 2500000 steps (86%)
21:35:48:WU00:FS01:0x21:Completed 2175000 out of 2500000 steps (87%)
21:36:45:WU00:FS01:0x21:Completed 2200000 out of 2500000 steps (88%)
21:37:52:WU00:FS01:0x21:Completed 2225000 out of 2500000 steps (89%)
21:38:49:WU00:FS01:0x21:Completed 2250000 out of 2500000 steps (90%)
21:39:45:WU00:FS01:0x21:Completed 2275000 out of 2500000 steps (91%)
21:40:42:WU00:FS01:0x21:Completed 2300000 out of 2500000 steps (92%)
21:41:49:WU00:FS01:0x21:Completed 2325000 out of 2500000 steps (93%)
21:42:46:WU00:FS01:0x21:Completed 2350000 out of 2500000 steps (94%)
21:43:42:WU00:FS01:0x21:Completed 2375000 out of 2500000 steps (95%)
21:44:39:WU00:FS01:0x21:Completed 2400000 out of 2500000 steps (96%)
21:45:46:WU00:FS01:0x21:Completed 2425000 out of 2500000 steps (97%)
21:46:43:WU00:FS01:0x21:Completed 2450000 out of 2500000 steps (98%)
21:47:39:WU00:FS01:0x21:Completed 2475000 out of 2500000 steps (99%)
21:47:54:WARNING:WU01:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known.
21:47:54:ERROR:WU01:FS01:Exception: Could not get an assignment
21:47:54:WU01:FS01:Connecting to 171.67.108.45:80
21:47:55:WU01:FS01:Assigned to work server 171.67.108.159
21:47:55:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:1:GP104 [GeForce GTX 1070] from 171.67.108.159
21:47:55:WU01:FS01:Connecting to 171.67.108.159:8080
21:47:56:WU01:FS01:Downloading 23.59MiB
21:48:36:WU00:FS01:0x21:Completed 2500000 out of 2500000 steps (100%)
21:48:46:WU00:FS01:0x21:Saving result file logfile_01.txt
21:48:46:WU00:FS01:0x21:Saving result file checkpointState.xml
21:48:46:WU00:FS01:0x21:Saving result file checkpt.crc
21:48:46:WU00:FS01:0x21:Saving result file log.txt
21:48:46:WU00:FS01:0x21:Saving result file positions.xtc
21:48:46:WU00:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
21:48:47:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:48:47:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
21:48:47:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
21:48:47:WU00:FS01:Connecting to 171.67.108.105:8080
21:49:42:WU01:FS01:Download 0.53%
21:51:28:WU01:FS01:Download 0.79%
21:51:34:WU01:FS01:Download 14.84%
21:51:40:WU01:FS01:Download 43.45%
21:51:46:WU01:FS01:Download 72.07%
21:51:51:WU01:FS01:Download complete
21:51:52:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9180 run:7 clone:8 gen:82 core:0x21 unit:0x00000061ab436c9f57bdce059b60e62e
21:51:52:WU01:FS01:Starting
21:51:52:WU01:FS01:Running FahCore: "E:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Kris/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 2732 -checkpoint 3 -gpu 0 -gpu-vendor nvidia
21:51:52:WU01:FS01:Started FahCore on PID 5204
21:51:52:WU01:FS01:Core PID:5500
21:51:52:WU01:FS01:FahCore 0x21 started
21:51:53:WU01:FS01:0x21:*********************** Log Started 2016-10-09T21:51:53Z ***********************
21:51:53:WU01:FS01:0x21:Project: 9180 (Run 7, Clone 8, Gen 82)
21:51:53:WU01:FS01:0x21:Unit: 0x00000061ab436c9f57bdce059b60e62e
21:51:53:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
21:51:53:WU01:FS01:0x21:Machine: 1
21:51:53:WU01:FS01:0x21:Reading tar file core.xml
21:51:53:WU01:FS01:0x21:Reading tar file integrator.xml
21:51:53:WU01:FS01:0x21:Reading tar file state.xml
21:51:53:WU01:FS01:0x21:Reading tar file system.xml
21:51:53:WU01:FS01:0x21:Digital signatures verified
21:51:53:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
21:51:53:WU01:FS01:0x21:Version 0.0.17
21:52:02:WU00:FS01:Upload 0.52%
21:52:02:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
21:52:02:WU00:FS01:Trying to send results to collection server
21:52:02:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
21:52:02:WU00:FS01:Connecting to 171.67.108.46:8080
21:52:08:WU00:FS01:Upload 2.07%
21:52:14:WU00:FS01:Upload 5.19%
21:52:20:WU00:FS01:Upload 8.30%
21:52:23:WU01:FS01:0x21:Completed 0 out of 2500000 steps (0%)
21:52:23:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
21:52:26:WU00:FS01:Upload 11.41%
21:52:32:WU00:FS01:Upload 14.52%
21:52:38:WU00:FS01:Upload 17.63%
21:52:45:WU00:FS01:Upload 20.74%
21:53:06:WU00:FS01:Upload 21.26%
21:53:06:ERROR:WU00:FS01:Exception: Transfer failed
21:53:06:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
21:53:06:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
21:53:06:WU00:FS01:Connecting to 171.67.108.105:8080
21:53:27:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
21:53:27:WU00:FS01:Connecting to 171.67.108.105:80
21:53:40:WU01:FS01:0x21:Completed 25000 out of 2500000 steps (1%)
21:53:43:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.105:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
21:53:43:WU00:FS01:Trying to send results to collection server
21:53:43:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
21:53:43:WU00:FS01:Connecting to 171.67.108.46:8080
21:54:04:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
21:54:04:WU00:FS01:Connecting to 171.67.108.46:80
21:54:25:ERROR:WU00:FS01:Exception: Failed to connect to 171.67.108.46:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
21:54:25:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
21:54:25:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
21:54:25:WU00:FS01:Connecting to 171.67.108.105:8080
21:54:47:WU01:FS01:0x21:Completed 50000 out of 2500000 steps (2%)
21:55:00:WU00:FS01:Upload 0.52%
21:55:00:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
21:55:00:WU00:FS01:Trying to send results to collection server
21:55:00:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
21:55:00:WU00:FS01:Connecting to 171.67.108.46:8080
21:55:54:WU01:FS01:0x21:Completed 75000 out of 2500000 steps (3%)
21:56:12:WU00:FS01:Upload 0.52%
21:56:12:ERROR:WU00:FS01:Exception: Transfer failed
21:56:13:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
21:56:13:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
21:56:13:WU00:FS01:Connecting to 171.67.108.105:8080
21:56:19:WU00:FS01:Upload 3.11%
21:56:26:WU00:FS01:Upload 5.70%
21:56:32:WU00:FS01:Upload 8.82%
21:56:38:WU00:FS01:Upload 11.93%
21:56:50:WU00:FS01:Upload 15.04%
21:57:01:WU01:FS01:0x21:Completed 100000 out of 2500000 steps (4%)
21:57:12:WU00:FS01:Upload 17.11%
21:57:12:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
21:57:12:WU00:FS01:Trying to send results to collection server
21:57:13:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
21:57:13:WU00:FS01:Connecting to 171.67.108.46:8080
21:57:19:WU00:FS01:Upload 2.59%
21:57:25:WU00:FS01:Upload 5.70%
21:57:31:WU00:FS01:Upload 8.82%
21:57:37:WU00:FS01:Upload 11.93%
21:57:43:WU00:FS01:Upload 15.04%
21:57:59:WU00:FS01:Upload 17.63%
21:58:19:WU01:FS01:0x21:Completed 125000 out of 2500000 steps (5%)
21:58:28:WU00:FS01:Upload 19.71%
21:58:28:ERROR:WU00:FS01:Exception: Transfer failed
21:58:50:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
21:58:50:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
21:58:50:WU00:FS01:Connecting to 171.67.108.105:8080
21:59:11:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
21:59:11:WU00:FS01:Connecting to 171.67.108.105:80
21:59:26:WU01:FS01:0x21:Completed 150000 out of 2500000 steps (6%)
21:59:32:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.105:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
21:59:32:WU00:FS01:Trying to send results to collection server
21:59:32:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
21:59:32:WU00:FS01:Connecting to 171.67.108.46:8080
21:59:53:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
21:59:53:WU00:FS01:Connecting to 171.67.108.46:80
22:00:14:ERROR:WU00:FS01:Exception: Failed to connect to 171.67.108.46:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
22:00:33:WU01:FS01:0x21:Completed 175000 out of 2500000 steps (7%)
22:01:40:WU01:FS01:0x21:Completed 200000 out of 2500000 steps (8%)
22:02:59:WU01:FS01:0x21:Completed 225000 out of 2500000 steps (9%)
22:03:04:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
22:03:04:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
22:03:04:WU00:FS01:Connecting to 171.67.108.105:8080
22:03:25:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
22:03:25:WU00:FS01:Connecting to 171.67.108.105:80
22:03:27:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.105:80: No connection could be made because the target machine actively refused it.
22:03:27:WU00:FS01:Trying to send results to collection server
22:03:27:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
22:03:27:WU00:FS01:Connecting to 171.67.108.46:8080
22:03:37:WU00:FS01:Upload 0.52%
22:03:58:WU00:FS01:Upload 1.04%
22:03:58:ERROR:WU00:FS01:Exception: Transfer failed
22:04:06:WU01:FS01:0x21:Completed 250000 out of 2500000 steps (10%)
22:05:12:WU01:FS01:0x21:Completed 275000 out of 2500000 steps (11%)
22:06:18:WU01:FS01:0x21:Completed 300000 out of 2500000 steps (12%)
22:07:35:WU01:FS01:0x21:Completed 325000 out of 2500000 steps (13%)
22:08:41:WU01:FS01:0x21:Completed 350000 out of 2500000 steps (14%)
22:09:47:WU01:FS01:0x21:Completed 375000 out of 2500000 steps (15%)
22:09:55:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
22:09:55:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
22:09:55:WU00:FS01:Connecting to 171.67.108.105:8080
22:10:53:WU01:FS01:0x21:Completed 400000 out of 2500000 steps (16%)
22:11:02:WU00:FS01:Upload 0.52%
22:11:02:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
22:11:02:WU00:FS01:Trying to send results to collection server
22:11:02:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
22:11:02:WU00:FS01:Connecting to 171.67.108.46:8080
22:11:23:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
22:11:23:WU00:FS01:Connecting to 171.67.108.46:80
22:11:44:ERROR:WU00:FS01:Exception: Failed to connect to 171.67.108.46:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
22:12:11:WU01:FS01:0x21:Completed 425000 out of 2500000 steps (17%)
22:13:17:WU01:FS01:0x21:Completed 450000 out of 2500000 steps (18%)
22:14:24:WU01:FS01:0x21:Completed 475000 out of 2500000 steps (19%)
22:15:30:WU01:FS01:0x21:Completed 500000 out of 2500000 steps (20%)
22:16:49:WU01:FS01:0x21:Completed 525000 out of 2500000 steps (21%)
22:17:56:WU01:FS01:0x21:Completed 550000 out of 2500000 steps (22%)
22:19:03:WU01:FS01:0x21:Completed 575000 out of 2500000 steps (23%)
22:20:10:WU01:FS01:0x21:Completed 600000 out of 2500000 steps (24%)
22:21:01:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9175 run:2 clone:15 gen:77 core:0x21 unit:0x0000005bab436c6957b24c27c32c33e7
22:21:01:WU00:FS01:Uploading 12.05MiB to 171.67.108.105
22:21:01:WU00:FS01:Connecting to 171.67.108.105:8080
22:21:22:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
22:21:22:WU00:FS01:Connecting to 171.67.108.105:80
22:21:28:WU01:FS01:0x21:Completed 625000 out of 2500000 steps (25%)
22:21:29:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.105:80: No connection could be made because the target machine actively refused it.
22:21:29:WU00:FS01:Trying to send results to collection server
22:21:29:WU00:FS01:Uploading 12.05MiB to 171.67.108.46
22:21:29:WU00:FS01:Connecting to 171.67.108.46:8080
22:21:35:WU00:FS01:Upload 2.07%
22:21:41:WU00:FS01:Upload 5.19%
22:21:47:WU00:FS01:Upload 8.30%
22:21:53:WU00:FS01:Upload 11.41%
22:21:59:WU00:FS01:Upload 14.52%
22:22:05:WU00:FS01:Upload 17.63%
22:22:11:WU00:FS01:Upload 20.74%
22:22:17:WU00:FS01:Upload 23.85%
22:22:23:WU00:FS01:Upload 26.96%
22:22:29:WU00:FS01:Upload 30.08%
22:22:35:WU00:FS01:Upload 32.67%
22:22:35:WU01:FS01:0x21:Completed 650000 out of 2500000 steps (26%)
22:22:41:WU00:FS01:Upload 35.78%
22:22:47:WU00:FS01:Upload 38.89%
22:22:53:WU00:FS01:Upload 42.00%
22:22:59:WU00:FS01:Upload 45.11%
22:23:05:WU00:FS01:Upload 48.23%
22:23:11:WU00:FS01:Upload 51.34%
22:23:17:WU00:FS01:Upload 54.45%
22:23:23:WU00:FS01:Upload 57.56%
22:23:29:WU00:FS01:Upload 60.67%
22:23:35:WU00:FS01:Upload 63.78%
22:23:41:WU00:FS01:Upload 66.89%
22:23:42:WU01:FS01:0x21:Completed 675000 out of 2500000 steps (27%)
22:23:47:WU00:FS01:Upload 70.00%
22:23:53:WU00:FS01:Upload 73.12%
22:23:59:WU00:FS01:Upload 75.71%
22:24:05:WU00:FS01:Upload 78.82%
22:24:11:WU00:FS01:Upload 81.93%
22:24:17:WU00:FS01:Upload 85.04%
22:24:23:WU00:FS01:Upload 88.15%
22:24:29:WU00:FS01:Upload 91.27%
22:24:35:WU00:FS01:Upload 94.38%
22:24:41:WU00:FS01:Upload 97.49%
22:24:46:WU00:FS01:Upload complete
22:24:46:WU00:FS01:Server responded WORK_QUIT (404)
22:24:46:WARNING:WU00:FS01:Server did not like results, dumping
22:24:46:WU00:FS01:Cleaning up
22:24:48:WU01:FS01:0x21:Completed 700000 out of 2500000 steps (28%)
22:26:07:WU01:FS01:0x21:Completed 725000 out of 2500000 steps (29%)
22:27:13:WU01:FS01:0x21:Completed 750000 out of 2500000 steps (30%)
22:28:19:WU01:FS01:0x21:Completed 775000 out of 2500000 steps (31%)
22:29:25:WU01:FS01:0x21:Completed 800000 out of 2500000 steps (32%)
22:30:42:WU01:FS01:0x21:Completed 825000 out of 2500000 steps (33%)
22:31:48:WU01:FS01:0x21:Completed 850000 out of 2500000 steps (34%)
22:32:54:WU01:FS01:0x21:Completed 875000 out of 2500000 steps (35%)
22:34:00:WU01:FS01:0x21:Completed 900000 out of 2500000 steps (36%)
22:35:18:WU01:FS01:0x21:Completed 925000 out of 2500000 steps (37%)
22:36:26:WU01:FS01:0x21:Completed 950000 out of 2500000 steps (38%)
22:37:33:WU01:FS01:0x21:Completed 975000 out of 2500000 steps (39%)
22:38:39:WU01:FS01:0x21:Completed 1000000 out of 2500000 steps (40%)
22:39:57:WU01:FS01:0x21:Completed 1025000 out of 2500000 steps (41%)
22:41:03:WU01:FS01:0x21:Completed 1050000 out of 2500000 steps (42%)
22:42:09:WU01:FS01:0x21:Completed 1075000 out of 2500000 steps (43%)
22:43:15:WU01:FS01:0x21:Completed 1100000 out of 2500000 steps (44%)
22:44:33:WU01:FS01:0x21:Completed 1125000 out of 2500000 steps (45%)
22:45:39:WU01:FS01:0x21:Completed 1150000 out of 2500000 steps (46%)
22:46:45:WU01:FS01:0x21:Completed 1175000 out of 2500000 steps (47%)
22:47:51:WU01:FS01:0x21:Completed 1200000 out of 2500000 steps (48%)
22:49:09:WU01:FS01:0x21:Completed 1225000 out of 2500000 steps (49%)
22:50:15:WU01:FS01:0x21:Completed 1250000 out of 2500000 steps (50%)
22:51:21:WU01:FS01:0x21:Completed 1275000 out of 2500000 steps (51%)
22:52:27:WU01:FS01:0x21:Completed 1300000 out of 2500000 steps (52%)
22:53:45:WU01:FS01:0x21:Completed 1325000 out of 2500000 steps (53%)
22:54:51:WU01:FS01:0x21:Completed 1350000 out of 2500000 steps (54%)
22:55:57:WU01:FS01:0x21:Completed 1375000 out of 2500000 steps (55%)
22:57:03:WU01:FS01:0x21:Completed 1400000 out of 2500000 steps (56%)
22:58:21:WU01:FS01:0x21:Completed 1425000 out of 2500000 steps (57%)
22:59:27:WU01:FS01:0x21:Completed 1450000 out of 2500000 steps (58%)
23:00:33:WU01:FS01:0x21:Completed 1475000 out of 2500000 steps (59%)
23:01:39:WU01:FS01:0x21:Completed 1500000 out of 2500000 steps (60%)
23:02:57:WU01:FS01:0x21:Completed 1525000 out of 2500000 steps (61%)
23:04:03:WU01:FS01:0x21:Completed 1550000 out of 2500000 steps (62%)
23:05:09:WU01:FS01:0x21:Completed 1575000 out of 2500000 steps (63%)
23:06:15:WU01:FS01:0x21:Completed 1600000 out of 2500000 steps (64%)
23:07:33:WU01:FS01:0x21:Completed 1625000 out of 2500000 steps (65%)
23:08:39:WU01:FS01:0x21:Completed 1650000 out of 2500000 steps (66%)
23:09:45:WU01:FS01:0x21:Completed 1675000 out of 2500000 steps (67%)
23:10:51:WU01:FS01:0x21:Completed 1700000 out of 2500000 steps (68%)
Ok i've just had another wu finish and this one uploaded fine but it went to a different server (171.67.108.159)
Code: Select all
23:48:08:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
23:48:08:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:48:09:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9180 run:7 clone:8 gen:82 core:0x21 unit:0x00000061ab436c9f57bdce059b60e62e
23:48:09:WU01:FS01:Uploading 14.13MiB to 171.67.108.159
23:48:09:WU01:FS01:Connecting to 171.67.108.159:8080
23:48:09:WU00:FS01:Starting
23:48:09:WU00:FS01:Running FahCore: "E:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Kris/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 2732 -checkpoint 3 -gpu 0 -gpu-vendor nvidia
23:48:09:WU00:FS01:Started FahCore on PID 5176
23:48:10:WU00:FS01:Core PID:6636
23:48:10:WU00:FS01:FahCore 0x21 started
23:48:11:WU00:FS01:0x21:*********************** Log Started 2016-10-09T23:48:11Z ***********************
23:48:11:WU00:FS01:0x21:Project: 9197 (Run 2, Clone 0, Gen 24)
23:48:11:WU00:FS01:0x21:Unit: 0x0000001eab40415c57cb3fcd9779aee5
23:48:11:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
23:48:11:WU00:FS01:0x21:Machine: 1
23:48:11:WU00:FS01:0x21:Reading tar file core.xml
23:48:11:WU00:FS01:0x21:Reading tar file system.xml
23:48:11:WU00:FS01:0x21:Reading tar file integrator.xml
23:48:11:WU00:FS01:0x21:Reading tar file state.xml
23:48:11:WU00:FS01:0x21:Digital signatures verified
23:48:11:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
23:48:11:WU00:FS01:0x21:Version 0.0.17
23:48:15:WU01:FS01:Upload 2.65%
23:48:21:WU01:FS01:Upload 6.63%
23:48:27:WU01:FS01:Upload 10.17%
23:48:33:WU01:FS01:Upload 13.71%
23:48:36:WU00:FS01:0x21:Completed 0 out of 2500000 steps (0%)
23:48:36:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:48:39:WU01:FS01:Upload 17.25%
23:48:45:WU01:FS01:Upload 19.90%
23:48:51:WU01:FS01:Upload 22.55%
23:48:57:WU01:FS01:Upload 25.21%
23:49:03:WU01:FS01:Upload 27.86%
23:49:09:WU01:FS01:Upload 30.52%
23:49:15:WU01:FS01:Upload 33.17%
23:49:21:WU01:FS01:Upload 35.82%
23:49:27:WU01:FS01:Upload 38.92%
23:49:33:WU01:FS01:Upload 41.57%
23:49:38:WU00:FS01:0x21:Completed 25000 out of 2500000 steps (1%)
23:49:39:WU01:FS01:Upload 44.22%
23:49:45:WU01:FS01:Upload 46.88%
23:49:51:WU01:FS01:Upload 49.53%
23:49:57:WU01:FS01:Upload 52.19%
23:50:03:WU01:FS01:Upload 54.84%
23:50:09:WU01:FS01:Upload 57.93%
23:50:15:WU01:FS01:Upload 60.59%
23:50:21:WU01:FS01:Upload 63.24%
23:50:27:WU01:FS01:Upload 65.90%
23:50:32:WU00:FS01:0x21:Completed 50000 out of 2500000 steps (2%)
23:50:33:WU01:FS01:Upload 68.55%
23:50:39:WU01:FS01:Upload 71.20%
23:50:45:WU01:FS01:Upload 73.86%
23:50:51:WU01:FS01:Upload 76.51%
23:50:57:WU01:FS01:Upload 79.60%
23:51:03:WU01:FS01:Upload 82.26%
23:51:09:WU01:FS01:Upload 84.91%
23:51:15:WU01:FS01:Upload 87.57%
23:51:21:WU01:FS01:Upload 90.22%
23:51:26:WU00:FS01:0x21:Completed 75000 out of 2500000 steps (3%)
23:51:27:WU01:FS01:Upload 92.87%
23:51:33:WU01:FS01:Upload 95.53%
23:51:39:WU01:FS01:Upload 98.18%
23:51:43:WU01:FS01:Upload complete
23:51:43:WU01:FS01:Server responded WORK_ACK (400)
23:51:43:WU01:FS01:Final credit estimate, 41853.00 points
23:51:43:WU01:FS01:Cleaning up
Re: 171.67.108.105 / 46
Posted: Mon Oct 10, 2016 12:13 am
by Joe_H
Yes, different projects, different servers.
By default the client keeps the current log in the FAH data directory and the previous 16 logs in a folder there as well. If your current log does not have an example of the upload problem, could you check the recent older logs and post an example?
Re: 171.67.108.105 / 46
Posted: Mon Oct 10, 2016 12:20 am
by soa_rru
The 1st log i posted shows the upload problems. I've checked back through my previous logs and there's maybe 7 or 8 times where i've encountered the same problem and its always the same servers 171.67.108.105 / 46
Re: 171.67.108.105 / 46
Posted: Tue Oct 11, 2016 12:29 am
by bruce
I'm checking with msultan to see if he can tell what's causing your problems.
.
Re: 171.67.108.105 / 46
Posted: Tue Oct 11, 2016 7:46 am
by soa_rru
bruce wrote:I'm checking with msultan to see if he can tell what's causing your problems.
.
Thanks.
Re: 171.67.108.105 / 46
Posted: Wed Oct 12, 2016 5:47 am
by msultan
Thanks for the report @soa_rru and thanks @bruce for sending me an email about this. I am not completely sure what is going on. The WS has ~100TB of free space so i dont think its a space issue. It could be either a networking issue or disk write permissions or it could be something else. However, work from several other WUs has been accepted successfully(even as wrote this email) so it could potentially be a stochastic problem.
I will continue to investigate this, and I have restarted the WS as a precaution. Please let me know if/when there are any further problems.
Re: 171.67.108.105 / 46
Posted: Tue Oct 18, 2016 5:58 pm
by soa_rru
Unfortunately i'm still having problems. Over an hour now trying to upload to said servers. I'm getting to a point if i see a WU which uses said servers i'm just going to dump the WU.
So it finally uploaded after 1.5hrs, i was half expecting the server to throw a hissy fit again and dump the results but it didn't, the only downside to a long upload is it lost around 16k in pts.
Re: 171.67.108.105 / 46
Posted: Tue Oct 18, 2016 7:47 pm
by bruce
@soa_rru
What does traceroute (Linux) or tractrt (Windows) to that server show?
Re: 171.67.108.105 / 46
Posted: Wed Oct 19, 2016 7:53 am
by soa_rru
I've been looking back at the log and the server it finally uploaded to last night was 171.67.108.
104 BUT checking back further 171.67.108.
104 / 45 were also failing to receive my uploads. Normally if the WU server is listed as
say 171.67.108.105 / 46 it will normally bounce between the two but last night was the 1st time i'd seen it bring another ###.##.###.104 / 45 into the mix.
The next WU i completed after that awkward one uploaded straight away and that went to 171.67.108.104 which was playing up some 4-5hrs earlier
Anyway here's a tracert
Code: Select all
Tracing route to vspe5.stanford.edu [171.67.108.105]
over a maximum of 30 hops:
1 1 ms 1 ms 1 ms 192.168.1.1
2 8 ms 7 ms 7 ms 213.1.113.85
3 8 ms 8 ms 7 ms 213.1.128.78
4 12 ms 12 ms 11 ms 213.1.78.126
5 13 ms 12 ms 13 ms 87.237.20.136
6 232 ms 18 ms 34 ms 40ge1-3.core1.lon2.he.net [195.66.224.21]
7 88 ms 87 ms 86 ms 100ge1-1.core1.nyc4.he.net [72.52.92.166]
8 142 ms 142 ms 146 ms 100ge14-2.core1.sjc2.he.net [184.105.81.213]
9 145 ms 149 ms 149 ms 10ge3-3.core1.pao1.he.net [72.52.92.69]
10 143 ms 143 ms 143 ms stanford-university.10gigabitethernet1-4.core1.p
ao1.he.net [216.218.209.118]
11 155 ms 154 ms 166 ms west-rtr-vlan8.SUNet [171.64.255.193]
12 156 ms 155 ms 155 ms vspe5.stanford.edu [171.67.108.105]
Trace complete.
Tracing route to VSP14.stanford.edu [171.67.108.46]
over a maximum of 30 hops:
1 1 ms 1 ms 3 ms 192.168.1.1
2 7 ms 6 ms 8 ms 213.1.113.85
3 7 ms 8 ms 8 ms 213.1.128.94
4 12 ms 11 ms 12 ms 213.1.79.6
5 13 ms 15 ms 12 ms 87.237.20.136
6 18 ms 23 ms 13 ms 40ge1-3.core1.lon2.he.net [195.66.224.21]
7 86 ms 80 ms 81 ms 100ge1-1.core1.nyc4.he.net [72.52.92.166]
8 150 ms 149 ms 149 ms 100ge14-2.core1.sjc2.he.net [184.105.81.213]
9 143 ms 144 ms 147 ms 10ge3-3.core1.pao1.he.net [72.52.92.69]
10 145 ms 145 ms 146 ms stanford-university.10gigabitethernet1-4.core1.p
ao1.he.net [216.218.209.118]
11 157 ms 157 ms 156 ms west-rtr-vlan8.SUNet [171.64.255.193]
12 150 ms 150 ms 151 ms VSP14.stanford.edu [171.67.108.46]
Trace complete.
Checking back further still & yesterday 128.252.203.2 & 140.163.4.243 were also giving me the same problem which has me wondering if the problem might actually be at my end?
I'm at a loss
Re: 171.67.108.105 / 46
Posted: Wed Oct 19, 2016 5:31 pm
by bruce
6 232 ms 18 ms 34 ms 40ge1-3.core1.lon2.he.net [195.66.224.21]
This is the only thing in that traceroute that's a little out of place but I'm not sure how to interpret it. Why would that router take 10x a long to figure out the routing to stanford than the next two pings?
Re: 171.67.108.105 / 46
Posted: Wed Oct 19, 2016 5:34 pm
by bruce
Most (but not all) stanford servers have a 15 minute timeout ... meaning if it takes more 15 minutes to upload a WU, the internet link is going to be disconnected. Look over all of your attempted uploads and see if that explains any of them.
(The 15 minute timeout is fine for not a problem except for the biggest WU results over the slowest connections. If anybody is still using a 56kb or slower modem, they should reduce the size of the max-packet-size setting.)
Re: 171.67.108.105 / 46
Posted: Wed Nov 16, 2016 5:31 pm
by yalexey
I have a problems with the same subnet now. One WU able to upload more than a day. The other two were downloaded for 10-20 minutes. The card is idle.
This was before too. I think, I will just deny connections to server in this network. On the other servers I do not have such problems.