171.64.65.56 ???

[314159]

Just received one from .54 here too.
I think that our good friend "The Doctor" is screwing around.

Moving to the Commonwealth of Virginia can bring on that type of behavior, (which he has apparently done per info on the web)

If you're reading this, Peter, congratulations on your new position (and note my sig line).

[kasson]

Thanks, and noted.
We implemented a new script to help keep server .56 running happily. Thus far, it seems to be working. You may notice some occasional slowdowns, but it should get fixed whenever we hit too much of a crawl...

[toTOW]

Something seems wrong with the network speed and reliability (I saw a lot of holes of inactivity while uploading) :

Code: Select all

[17:54:10] + Attempting to send results [October 28 17:54:10 UTC]
[17:54:10] - Reading file work/wuresults_06.dat from core
[17:54:10]   (Read 43235852 bytes from disk)
[17:54:10] Connecting to http://171.64.65.56:8080/
[19:03:59] Posted data.
[19:04:00] Initial: 0000; - Uploaded at ~10 kB/s
[19:04:00] - Averaged speed for that direction ~45 kB/s
[19:04:00] + Results successfully sent
[19:04:00] Thank you for your contribution to Folding@Home.

[314159]

Server needs reset PLEASE. I have MANY WUs queued.

NET LOAD again?

"Good Doctor"? Are you around?

.xx.54 ALSO has problems.

[314159]

Last one is sending but will take all night - almost no bandwidth. (and NOT a local problem)
NET LOAD still high.
.54 still has similar problems.
With 10 to complete in the next few hours and 4 queued and hopefully (?) going to send in 5-6 hours, this is a real mess.

Oh well....

[weedacres]

Same problems here. 10-50kbps upload, stalls and fails on upload.

[314159]

Try now.

It appears that the problems had been somewhat widespread but many are now corrected or in process.

Fold on

[One_Box]

I have been trying to upload 2 completed WUs to this server one of which has been outstanding for 2 days the other just completed.

Getting rather fed up with work server issues (both SMP and GPU3) which seem unreliable at best compared to 2 GPU2 clients I run which seem to chug along just fine.

It is really frustrating.

[bruce]

I have been trying to upload 2 completed WUs to this server one of which has been outstanding for 2 days the other just completed.

Getting rather fed up with work server issues (both SMP and GPU3) which seem unreliable at best compared to 2 GPU2 clients I run which seem to chug along just fine.

It is really frustrating.

Please post FAHlog.txt showing the problems you're having. I can see nothing wrong with that server. Are you sure the problem isn't something related to your security software?

[One_Box]

Please post FAHlog.txt showing the problems you're having. I can see nothing wrong with that server. Are you sure the problem isn't something related to your security software?

I believe the problem is not with my computer, and I enclose a small portion of the log file for your inspection.
Both SMP WUs have now uploaded successfully but it took approx 10hours for this to happen after completion of the 2nd unit!!
You will notice that one unit failed to upload 35 times over a period of about 2 days.
At the time of the 10 hour period it was the only computer attached to my router.
I have noticed similar problems in the recent past with GPU3 WUs although GPU2 units seem to upload very reliably.
I have swapped Fermi cards for GTS 250/GT 240 and immediately the uploading problems go away.
I have confirmed with my ISP that they are not blocking any of the server addresses and I believe my firewall and antivirus software is configured correctly.

If you are able to shed any light on these ongoing problems it would be much appreciated


[14:39:33] + Attempting to send results [November 11 14:39:33 UTC]
[14:39:33] - Reading file work/wuresults_07.dat from core
[14:39:33] (Read 43603905 bytes from disk)
[14:39:33] Connecting to http://171.67.108.25:8080/
[14:39:33] - Couldn't send HTTP request to server
[14:39:33] (Got status 503)
[14:39:33] + Could not connect to Work Server (results)
[14:39:33] (171.67.108.25:8080)
[14:39:33] + Retrying using alternative port
[14:39:33] Connecting to http://171.67.108.25:80/
[14:39:34] - Couldn't send HTTP request to server
[14:39:34] (Got status 503)
[14:39:34] + Could not connect to Work Server (results)
[14:39:34] (171.67.108.25:80)
[14:39:34] Could not transmit unit 07 to Collection server; keeping in queue.
[14:39:34] + Sent 0 of 1 completed units to the server
[14:40:04] Trying to send all finished work units
[14:40:04] Project: 6701 (Run 65, Clone 7, Gen 40)


[14:40:04] + Attempting to send results [November 11 14:40:04 UTC]
[14:40:04] - Reading file work/wuresults_07.dat from core
[14:40:04] (Read 43603905 bytes from disk)
[14:40:04] Connecting to http://171.64.65.56:8080/
[14:42:01] - Couldn't send HTTP request to server
[14:42:01] + Could not connect to Work Server (results)
[14:42:01] (171.64.65.56:8080)
[14:42:01] + Retrying using alternative port
[14:42:01] Connecting to http://171.64.65.56:80/
[14:43:39] - Couldn't send HTTP request to server
[14:43:39] + Could not connect to Work Server (results)
[14:43:39] (171.64.65.56:80)
[14:43:39] - Error: Could not transmit unit 07 (completed November 9) to work server.
[14:43:39] - 35 failed uploads of this unit.


[14:43:39] + Attempting to send results [November 11 14:43:39 UTC]
[14:43:39] - Reading file work/wuresults_07.dat from core
[14:43:39] (Read 43603905 bytes from disk)
[14:43:39] Connecting to http://171.67.108.25:8080/
[14:43:39] - Couldn't send HTTP request to server
[14:43:39] (Got status 503)
[14:43:39] + Could not connect to Work Server (results)
[14:43:39] (171.67.108.25:8080)
[14:43:39] + Retrying using alternative port
[14:43:39] Connecting to http://171.67.108.25:80/
[14:45:36] - Couldn't send HTTP request to server
[14:45:36] + Could not connect to Work Server (results)
[14:45:36] (171.67.108.25:80)
[14:45:36] Could not transmit unit 07 to Collection server; keeping in queue.
[14:45:36] + Sent 0 of 1 completed units to the server
[14:46:06] Trying to send all finished work units
[14:46:06] Project: 6701 (Run 65, Clone 7, Gen 40)


[14:46:06] + Attempting to send results [November 11 14:46:06 UTC]
[14:46:06] - Reading file work/wuresults_07.dat from core
[14:46:06] (Read 43603905 bytes from disk)
[14:46:06] Connecting to http://171.64.65.56:8080/
[15:03:44] Posted data.
[15:03:44] Initial: 0000; - Uploaded at ~40 kB/s
[15:03:45] - Averaged speed for that direction ~38 kB/s
[15:03:45] + Results successfully sent
[15:03:45] Thank you for your contribution to Folding@Home.
[15:03:45] + Number of Units Completed: 17

[15:03:46] + Sent 1 of 1 completed units to the server
[15:03:46] + -oneunit flag given and have now finished a unit. Exiting.***** Got a SIGTERM signal (2)
[15:03:46] Killing all core threads
[15:03:46] Could not get process id information. Please kill core process manually

Folding@Home Client Shutdown.

[bruce]

The Pande Group has been upgrading the server software. If you read the annoucements for the past several weeks, you'll see servers taken off-line and coming back on-line from time to time. One of the known problems with the old server code is that the NET LOAD gradually increases until the server gets unable to continue handling new connections. The new code has a fix for that problem.

As near as I can tell, the SMP servers have not yet been updated so until that happens, those servers are less reliable than they should be. I have no information about when that upgrade is planned.

If you need to discuss an issue about a GPU3 server, that's off-topic here since this topic is about 171.64.65.56.

[One_Box]

The Pande Group has been upgrading the server software. If you read the annoucements for the past several weeks, you'll see servers taken off-line and coming back on-line from time to time. One of the known problems with the old server code is that the NET LOAD gradually increases until the server gets unable to continue handling new connections. The new code has a fix for that problem.

As near as I can tell, the SMP servers have not yet been updated so until that happens, those servers are less reliable than they should be. I have no information about when that upgrade is planned.

If you need to discuss an issue about a GPU3 server, that's off-topic here since this topic is about 171.64.65.56.

Thank you for the info re the servers.

Will try to be more disciplined with my posts in future.

Lets hope ALL the servers get updated soon.

[sswilson]

Not sure if this is related or not as I'm not even getting to the IP assignment. It's just hanging at "attempting to send results".

I've tried using the -send all, as well as rebooting to no avail.

If somebody can confirm that this is related (i.e. by knowing what server a 6701 would typically upload to) I'll just sit back and go with the flow....

Code: Select all

--- Opening Log file [November 14 14:51:55 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit 

[14:51:55] - Ask before connecting: No
[14:51:55] - User name: sswilson (Team 54196)
[14:51:55] - User ID: ################
[14:51:55] - Machine ID: 8
[14:51:55] 
[14:51:55] Loaded queue successfully.
[14:51:55] 
[14:51:55] + Processing work unit
[14:51:55] Core required: FahCore_a3.exe
[14:51:55] Core found.
[14:51:55] Working on queue slot 00 [November 14 14:51:55 UTC]
[14:51:55] + Working ...
[14:51:55] 
[14:51:55] *------------------------------*
[14:51:55] Folding@Home Gromacs SMP Core
[14:51:55] Version 2.22 (Mar 12, 2010)
[14:51:55] 
[14:51:55] Preparing to commence simulation
[14:51:55] - Ensuring status. Please wait.
[14:52:05] - Looking at optimizations...
[14:52:05] - Working with standard loops on this execution.
[14:52:05] - Previous termination of core was improper.
[14:52:05] - Files status OK
[14:52:05] - Expanded 763889 -> 1404481 (decompressed 183.8 percent)
[14:52:05] Called DecompressByteArray: compressed_data_size=763889 data_size=1404481, decompressed_data_size=1404481 diff=0
[14:52:05] - Digital signature verified
[14:52:05] 
[14:52:05] Project: 6701 (Run 78, Clone 0, Gen 74)
[14:52:05] 
[14:52:05] Entering M.D.
[14:52:11] Using Gromacs checkpoints
[14:52:11] Resuming from checkpoint
[14:52:11] Verified work/wudata_00.log
[14:52:11] Verified work/wudata_00.trr
[14:52:11] Verified work/wudata_00.xtc
[14:52:11] Verified work/wudata_00.edr
[14:52:11] Completed 1842050 out of 2000000 steps  (92%)
[15:00:15] Completed 1860000 out of 2000000 steps  (93%)
[15:08:33] Completed 1880000 out of 2000000 steps  (94%)
[15:17:01] Completed 1900000 out of 2000000 steps  (95%)
[15:25:32] Completed 1920000 out of 2000000 steps  (96%)
[15:33:45] Completed 1940000 out of 2000000 steps  (97%)
[15:41:54] Completed 1960000 out of 2000000 steps  (98%)
[15:50:06] Completed 1980000 out of 2000000 steps  (99%)
[15:58:19] Completed 2000000 out of 2000000 steps  (100%)
[15:58:20] DynamicWrapper: Finished Work Unit: sleep=10000
[15:58:30] 
[15:58:30] Finished Work Unit:
[15:58:30] - Reading up to 687408 from "work/wudata_00.trr": Read 687408
[15:58:30] trr file hash check passed.
[15:58:30] - Reading up to 42617212 from "work/wudata_00.xtc": Read 42617212
[15:58:30] xtc file hash check passed.
[15:58:30] edr file hash check passed.
[15:58:30] logfile size: 286540
[15:58:30] Leaving Run
[15:58:34] - Writing 43593496 bytes of core data to disk...
[15:58:35]   ... Done.
[15:58:37] - Shutting down core
[15:58:37] 
[15:58:37] Folding@home Core Shutdown: FINISHED_UNIT
[15:58:40] CoreStatus = 64 (100)
[15:58:40] Sending work to server
[15:58:40] Project: 6701 (Run 78, Clone 0, Gen 74)


[15:58:40] + Attempting to send results [November 14 15:58:40 UTC]

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:05:08 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit 

[16:05:08] - Ask before connecting: No
[16:05:08] - User name: sswilson (Team 54196)
[16:05:08] - User ID: ################
[16:05:08] - Machine ID: 8
[16:05:08] 
[16:05:08] Loaded queue successfully.
[16:05:08] - Preparing to get new work unit...
[16:05:08] Project: 6701 (Run 78, Clone 0, Gen 74)
[16:05:08] Cleaning up work directory


[16:05:08] + Attempting to send results [November 14 16:05:08 UTC]
[16:05:08] + Attempting to get work packet
[16:05:08] Passkey found
[16:05:08] - Connecting to assignment server
[16:05:09] - Successful: assigned to (171.64.65.54).
[16:05:09] + News From Folding@Home: Welcome to Folding@Home
[16:05:10] Loaded queue successfully.
[16:05:21] + Closed connections
[16:05:21] 
[16:05:21] + Processing work unit
[16:05:21] Core required: FahCore_a3.exe
[16:05:21] Core found.
[16:05:21] Working on queue slot 01 [November 14 16:05:21 UTC]
[16:05:21] + Working ...
[16:05:21] 
[16:05:21] *------------------------------*
[16:05:21] Folding@Home Gromacs SMP Core
[16:05:21] Version 2.22 (Mar 12, 2010)
[16:05:21] 
[16:05:21] Preparing to commence simulation
[16:05:21] - Looking at optimizations...
[16:05:21] - Created dyn
[16:05:21] - Files status OK
[16:05:22] - Expanded 1767329 -> 2257001 (decompressed 127.7 percent)
[16:05:22] Called DecompressByteArray: compressed_data_size=1767329 data_size=2257001, decompressed_data_size=2257001 diff=0
[16:05:22] - Digital signature verified
[16:05:22] 
[16:05:22] Project: 6060 (Run 0, Clone 198, Gen 99)
[16:05:22] 
[16:05:22] Assembly optimizations on if available.
[16:05:22] Entering M.D.
[16:05:28] Completed 0 out of 500000 steps  (0%)
[16:08:56] Completed 5000 out of 500000 steps  (1%)

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:12:35 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit -send all 

[16:12:35] - Ask before connecting: No
[16:12:35] - User name: sswilson (Team 54196)
[16:12:35] - User ID: ################
[16:12:35] - Machine ID: 8
[16:12:35] 
[16:12:35] Loaded queue successfully.
[16:12:35] Attempting to return result(s) to server...
[16:12:35] Project: 6701 (Run 78, Clone 0, Gen 74)


[16:12:35] + Attempting to send results [November 14 16:12:35 UTC]

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:20:05 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit -send all 

[16:20:05] - Ask before connecting: No
[16:20:05] - User name: sswilson (Team 54196)
[16:20:05] - User ID: ################
[16:20:05] - Machine ID: 8
[16:20:05] 
[16:20:05] Loaded queue successfully.
[16:20:05] Attempting to return result(s) to server...
[16:20:05] Project: 6701 (Run 78, Clone 0, Gen 74)


[16:20:05] + Attempting to send results [November 14 16:20:05 UTC]

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:23:52 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit -send all 

[16:23:52] - Ask before connecting: No
[16:23:52] - User name: sswilson (Team 54196)
[16:23:52] - User ID: ################
[16:23:52] - Machine ID: 8
[16:23:52] 
[16:23:52] Loaded queue successfully.
[16:23:52] Attempting to return result(s) to server...
[16:23:52] Project: 6701 (Run 78, Clone 0, Gen 74)


[16:23:52] + Attempting to send results [November 14 16:23:52 UTC]

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:27:49 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit -send all 

[16:27:49] - Ask before connecting: No
[16:27:49] - User name: sswilson (Team 54196)
[16:27:49] - User ID: ################
[16:27:49] - Machine ID: 8
[16:27:49] 
[16:27:49] Loaded queue successfully.
[16:27:49] Attempting to return result(s) to server...
[16:27:49] Project: 6701 (Run 78, Clone 0, Gen 74)


[16:27:49] + Attempting to send results [November 14 16:27:49 UTC]

Folding@Home Client Shutdown at user request.

Folding@Home Client Shutdown.


--- Opening Log file [November 14 16:42:53 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.29

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: E:\FAH SMP2
Executable: E:\FAH SMP2\[email protected]
Arguments: -advmethods -smp -oneunit -send all 

[16:42:53] - Ask before connecting: No
[16:42:53] - User name: sswilson (Team 54196)
[16:42:53] - User ID: ################
[16:42:53] - Machine ID: 8
[16:42:53] 
[16:42:53] Loaded queue successfully.
[16:42:53] Attempting to return result(s) to server...
[16:42:53] Project: 6701 (Run 78, Clone 0, Gen 74)


[16:42:53] + Attempting to send results [November 14 16:42:53 UTC]

EDIT: NVM..... I guess it just needed to be threatened..... Looks like it uploaded while I was posting this.....

[dutchmm]

From my logs (times are UTC)

Code: Select all

[02:01:08] Folding@home Core Shutdown: FINISHED_UNIT
[02:01:12] CoreStatus = 64 (100)
[02:01:12] Sending work to server
[02:01:12] Project: 6701 (Run 39, Clone 34, Gen 39)


[02:01:12] + Attempting to send results [November 15 02:01:12 UTC]
[02:01:46] - Couldn't send HTTP request to server
[02:01:46] + Could not connect to Work Server (results)
[02:01:46]     (171.64.65.56:8080)
[02:01:46] + Retrying using alternative port
[02:01:49] - Couldn't send HTTP request to server
[02:01:49] + Could not connect to Work Server (results)
[02:01:49]     (171.64.65.56:80)
[02:01:49] - Error: Could not transmit unit 06 (completed November 15) to work server.
[02:01:49]   Keeping unit 06 in queue.
[02:01:49] Project: 6701 (Run 39, Clone 34, Gen 39)


[02:01:49] + Attempting to send results [November 15 02:01:49 UTC]
[02:01:49] - Couldn't send HTTP request to server
[02:01:49] + Could not connect to Work Server (results)
[02:01:49]     (171.64.65.56:8080)
[02:01:49] + Retrying using alternative port
[02:01:50] - Couldn't send HTTP request to server
[02:01:50] + Could not connect to Work Server (results)
[02:01:50]     (171.64.65.56:80)
[02:01:50] - Error: Could not transmit unit 06 (completed November 15) to work server.


[02:01:50] + Attempting to send results [November 15 02:01:50 UTC]
[02:02:38] - Couldn't send HTTP request to server
[02:02:38] + Could not connect to Work Server (results)
[02:02:38]     (171.67.108.25:8080)
[02:02:38] + Retrying using alternative port
[02:03:27] - Couldn't send HTTP request to server
[02:03:27] + Could not connect to Work Server (results)
[02:03:27]     (171.67.108.25:80)
[02:03:27]   Could not transmit unit 06 to Collection server; keeping in queue.
[02:03:27] - Preparing to get new work unit...
[02:03:27] Cleaning up work directory
[02:03:27] + Attempting to get work packet
[02:03:27] Passkey found
[02:03:27] - Connecting to assignment server
[02:03:29] - Successful: assigned to (171.64.65.56).
[02:03:29] + News From Folding@Home: Welcome to Folding@Home
[02:03:29] Loaded queue successfully.
[02:03:30] - Couldn't send HTTP request to server
[02:03:30] + Could not connect to Work Server
[02:03:30] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[02:03:48] + Attempting to get work packet
[02:03:48] Passkey found
[02:03:48] - Connecting to assignment server
[02:03:49] - Successful: assigned to (171.64.65.56).
[02:03:49] + News From Folding@Home: Welcome to Folding@Home
[02:03:49] Loaded queue successfully.
[02:03:49] - Couldn't send HTTP request to server
[02:03:49] + Could not connect to Work Server
[02:03:49] - Attempt #2  to get work failed, and no other work to do.
Waiting before retry.
[02:04:11] + Attempting to get work packet
[02:04:11] Passkey found
[02:04:11] - Connecting to assignment server
[02:04:12] - Successful: assigned to (171.64.65.56).
[02:04:12] + News From Folding@Home: Welcome to Folding@Home
[02:04:12] Loaded queue successfully.
[02:04:13] - Couldn't send HTTP request to server
[02:04:13] + Could not connect to Work Server
[02:04:13] - Attempt #3  to get work failed, and no other work to do.
Waiting before retry.
[02:04:42] + Attempting to get work packet
[02:04:42] Passkey found
[02:04:42] - Connecting to assignment server
[02:04:43] - Successful: assigned to (171.64.65.56).
[02:04:43] + News From Folding@Home: Welcome to Folding@Home
[02:04:43] Loaded queue successfully.
[02:04:48] Project: 6701 (Run 39, Clone 34, Gen 39)


[02:04:48] + Attempting to send results [November 15 02:04:48 UTC]
[02:21:24] + Results successfully sent
[02:21:24] Thank you for your contribution to Folding@Home.
[02:21:24] + Number of Units Completed: 70

It is now 07:18 the same day, and this WU does not show in my Stanford statistics. Can this be the result of the SMP reliability issues you have been discussing here?

[kasson]

That was actually a different issue. Unfortunately, the passkey/bonus information was lost for a set of work units returned over the weekend. We just did a recredit; the points will show up under your normal username + team with passkey 1. We apologize for the inconvenience (and for no bonus). The server had some problems over the weekend.