Project: 6900 (Run 30, Clone 19, Gen 2)

[MIBW]

Hi guys, I think this one was possibly due to me upping my memory speed a few days before, but just in case I thought I would report it...

I upped baseclock rather high, keeping GHz the same, to get mem speed up, but potentially introduced instability. It passed Memtest, intelburntest, and a few hours of a captured 6701 unit I use for stability testing, then went live. It finished a 6900 unit fine, then a 2685, then a 2692, (so about 48hrs ok) but a subsequent 6900 had Client-core communications error: ERROR 0xc0000029 at 86% of the way through, then also 19% of the way through the same unit. Interestingly I was assigned a 2686 after that without me having to do anything.

(The restart in the log was me doing other software things - I had not noticed it had an error at that point)

Code: Select all

[08:06:44] Project: 6900 (Run 30, Clone 19, Gen 2)
[08:06:44] 
[08:06:44] Assembly optimizations on if available.
[08:06:44] Entering M.D.
[08:06:52] Completed 0 out of 250000 steps  (0%)
[08:17:58] Completed 2500 out of 250000 steps  (1%)
[08:29:04] Completed 5000 out of 250000 steps  (2%)
[08:40:09] Completed 7500 out of 250000 steps  (3%)
[08:51:13] Completed 10000 out of 250000 steps  (4%)
[09:02:17] Completed 12500 out of 250000 steps  (5%)
[09:13:26] Completed 15000 out of 250000 steps  (6%)
[09:24:29] Completed 17500 out of 250000 steps  (7%)
[09:35:34] Completed 20000 out of 250000 steps  (8%)
[09:46:39] Completed 22500 out of 250000 steps  (9%)
[09:57:41] Completed 25000 out of 250000 steps  (10%)
[10:08:46] Completed 27500 out of 250000 steps  (11%)
[10:19:49] Completed 30000 out of 250000 steps  (12%)
[10:30:56] Completed 32500 out of 250000 steps  (13%)
[10:42:01] Completed 35000 out of 250000 steps  (14%)
[10:53:07] Completed 37500 out of 250000 steps  (15%)
[11:04:14] Completed 40000 out of 250000 steps  (16%)
[11:15:18] Completed 42500 out of 250000 steps  (17%)
[11:26:23] Completed 45000 out of 250000 steps  (18%)
[11:37:31] Completed 47500 out of 250000 steps  (19%)
[11:48:35] Completed 50000 out of 250000 steps  (20%)
[11:59:41] Completed 52500 out of 250000 steps  (21%)
[12:10:47] Completed 55000 out of 250000 steps  (22%)
[12:21:53] Completed 57500 out of 250000 steps  (23%)
[12:33:00] Completed 60000 out of 250000 steps  (24%)
[12:44:07] Completed 62500 out of 250000 steps  (25%)
[12:45:17] - Autosending finished units... [December 2 12:45:17 UTC]
[12:45:17] Trying to send all finished work units
[12:45:17] + No unsent completed units remaining.
[12:45:17] - Autosend completed
[12:55:14] Completed 65000 out of 250000 steps  (26%)
[13:06:28] Completed 67500 out of 250000 steps  (27%)
[13:17:36] Completed 70000 out of 250000 steps  (28%)
[13:28:43] Completed 72500 out of 250000 steps  (29%)
[13:39:47] Completed 75000 out of 250000 steps  (30%)
[13:50:53] Completed 77500 out of 250000 steps  (31%)
[14:01:57] Completed 80000 out of 250000 steps  (32%)
[14:13:02] Completed 82500 out of 250000 steps  (33%)
[14:24:05] Completed 85000 out of 250000 steps  (34%)
[14:35:13] Completed 87500 out of 250000 steps  (35%)
[14:46:20] Completed 90000 out of 250000 steps  (36%)
[14:57:25] Completed 92500 out of 250000 steps  (37%)
[15:08:31] Completed 95000 out of 250000 steps  (38%)
[15:19:38] Completed 97500 out of 250000 steps  (39%)
[15:30:40] Completed 100000 out of 250000 steps  (40%)
[15:41:46] Completed 102500 out of 250000 steps  (41%)
[15:52:54] Completed 105000 out of 250000 steps  (42%)
[16:04:00] Completed 107500 out of 250000 steps  (43%)
[16:15:06] Completed 110000 out of 250000 steps  (44%)
[16:26:12] Completed 112500 out of 250000 steps  (45%)
[16:37:19] Completed 115000 out of 250000 steps  (46%)
[16:48:25] Completed 117500 out of 250000 steps  (47%)
[16:59:31] Completed 120000 out of 250000 steps  (48%)
[17:10:37] Completed 122500 out of 250000 steps  (49%)
[17:21:42] Completed 125000 out of 250000 steps  (50%)
[17:32:49] Completed 127500 out of 250000 steps  (51%)
[17:43:56] Completed 130000 out of 250000 steps  (52%)
[17:55:03] Completed 132500 out of 250000 steps  (53%)
[18:06:08] Completed 135000 out of 250000 steps  (54%)
[18:17:18] Completed 137500 out of 250000 steps  (55%)
[18:28:24] Completed 140000 out of 250000 steps  (56%)
[18:39:28] Completed 142500 out of 250000 steps  (57%)
[18:45:17] - Autosending finished units... [December 2 18:45:17 UTC]
[18:45:17] Trying to send all finished work units
[18:45:17] + No unsent completed units remaining.
[18:45:17] - Autosend completed
[18:50:34] Completed 145000 out of 250000 steps  (58%)
[19:01:42] Completed 147500 out of 250000 steps  (59%)
[19:12:49] Completed 150000 out of 250000 steps  (60%)
[19:23:57] Completed 152500 out of 250000 steps  (61%)
[19:35:05] Completed 155000 out of 250000 steps  (62%)
[19:46:12] Completed 157500 out of 250000 steps  (63%)
[19:57:17] Completed 160000 out of 250000 steps  (64%)
[20:08:26] Completed 162500 out of 250000 steps  (65%)
[20:19:35] Completed 165000 out of 250000 steps  (66%)
[20:30:43] Completed 167500 out of 250000 steps  (67%)
[20:41:51] Completed 170000 out of 250000 steps  (68%)
[20:52:59] Completed 172500 out of 250000 steps  (69%)
[21:04:05] Completed 175000 out of 250000 steps  (70%)
[21:15:13] Completed 177500 out of 250000 steps  (71%)
[21:26:22] Completed 180000 out of 250000 steps  (72%)
[21:37:30] Completed 182500 out of 250000 steps  (73%)
[21:49:42] Completed 185000 out of 250000 steps  (74%)
[22:00:48] Completed 187500 out of 250000 steps  (75%)
[22:11:57] Completed 190000 out of 250000 steps  (76%)
[22:23:06] Completed 192500 out of 250000 steps  (77%)
[22:34:14] Completed 195000 out of 250000 steps  (78%)
[22:45:21] Completed 197500 out of 250000 steps  (79%)
[22:56:31] Completed 200000 out of 250000 steps  (80%)
[23:07:40] Completed 202500 out of 250000 steps  (81%)
[23:18:48] Completed 205000 out of 250000 steps  (82%)
[23:29:59] Completed 207500 out of 250000 steps  (83%)
[23:41:10] Completed 210000 out of 250000 steps  (84%)
[23:52:28] Completed 212500 out of 250000 steps  (85%)
[00:03:39] Completed 215000 out of 250000 steps  (86%)
[00:08:42] CoreStatus = C0000029 (-1073741783)
[00:08:42] Client-core communications error: ERROR 0xc0000029
[00:08:42] Deleting current work unit & continuing...
[00:09:24] Trying to send all finished work units
[00:09:24] + No unsent completed units remaining.
[00:09:24] - Preparing to get new work unit...
[00:09:24] Cleaning up work directory
[00:09:24] + Attempting to get work packet
[00:09:24] Passkey found
[00:09:24] - Will indicate memory of 12279 MB
[00:09:24] - Connecting to assignment server
[00:09:24] Connecting to http://assign.stanford.edu:8080/
[00:09:25] Posted data.
[00:09:25] Initial: ED82; - Successful: assigned to (130.237.232.141).
[00:09:25] + News From Folding@Home: Welcome to Folding@Home
[00:09:25] Loaded queue successfully.
[00:09:25] Sent data
[00:09:25] Connecting to http://130.237.232.141:8080/
[00:09:33] Posted data.
[00:09:33] Initial: 0000; - Receiving payload (expected size: 24857273)
[00:11:38] - Downloaded at ~194 kB/s
[00:11:38] - Averaged speed for that direction ~279 kB/s
[00:11:38] + Received work.
[00:11:38] + Closed connections
[00:11:43] 
[00:11:43] + Processing work unit
[00:11:43] Core required: FahCore_a3.exe
[00:11:43] Core found.
[00:11:43] Working on queue slot 00 [December 3 00:11:43 UTC]
[00:11:43] + Working ...
[00:11:43] - Calling '.\FahCore_a3.exe -dir work/ -nice 19 -suffix 00 -np 24 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 4076 -version 630'

[00:11:43] 
[00:11:43] *------------------------------*
[00:11:43] Folding@Home Gromacs SMP Core
[00:11:43] Version 2.22 (Mar 12, 2010)
[00:11:43] 
[00:11:43] Preparing to commence simulation
[00:11:43] - Assembly optimizations manually forced on.
[00:11:43] - Not checking prior termination.
[00:11:48] - Expanded 24856761 -> 30796293 (decompressed 123.8 percent)
[00:11:48] Called DecompressByteArray: compressed_data_size=24856761 data_size=30796293, decompressed_data_size=30796293 diff=0
[00:11:48] - Digital signature verified
[00:11:48] 
[00:11:48] Project: 6900 (Run 30, Clone 19, Gen 2)
[00:11:48] 
[00:11:48] Assembly optimizations on if available.
[00:11:48] Entering M.D.
[00:11:56] Completed 0 out of 250000 steps  (0%)
[00:23:07] Completed 2500 out of 250000 steps  (1%)
[00:34:21] Completed 5000 out of 250000 steps  (2%)
[00:46:55] Completed 7500 out of 250000 steps  (3%)
[00:47:42] - Autosending finished units... [December 3 00:47:42 UTC]
[00:47:42] Trying to send all finished work units
[00:47:42] + No unsent completed units remaining.
[00:47:42] - Autosend completed
[00:58:05] Completed 10000 out of 250000 steps  (4%)
[01:09:24] Completed 12500 out of 250000 steps  (5%)
[01:20:35] Completed 15000 out of 250000 steps  (6%)
[01:31:50] Completed 17500 out of 250000 steps  (7%)
[01:43:04] Completed 20000 out of 250000 steps  (8%)
[01:54:14] Completed 22500 out of 250000 steps  (9%)
[02:05:21] Completed 25000 out of 250000 steps  (10%)
[02:16:38] Completed 27500 out of 250000 steps  (11%)
[02:27:50] Completed 30000 out of 250000 steps  (12%)
[02:39:05] Completed 32500 out of 250000 steps  (13%)
[02:50:15] Completed 35000 out of 250000 steps  (14%)
[03:01:28] Completed 37500 out of 250000 steps  (15%)
[03:12:39] Completed 40000 out of 250000 steps  (16%)
[03:21:26] Killing all core threads
[03:21:26] Could not get process id information.  Please kill core process manually

Folding@Home Client Shutdown at user request.
[03:21:26] ***** Got a SIGTERM signal (2)
[03:21:26] Killing all core threads
[03:21:26] Could not get process id information.  Please kill core process manually

Folding@Home Client Shutdown.


--- Opening Log file [December 3 05:25:30 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.30

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Apps\FAH GPU Tracker V2\SMP
Executable: C:\Apps\FAH GPU Tracker V2\FAH.exe
Arguments: -forceasm -smp -verbosity 9 -bigadv 

[05:25:30] - Ask before connecting: No
[05:25:30] - User name: DigitalFX (Team 33)
[05:25:30] - User ID: AE604F034D9882E
[05:25:30] - Machine ID: 9
[05:25:30] 
[05:25:30] Loaded queue successfully.
[05:25:30] 
[05:25:30] - Autosending finished units... [December 3 05:25:30 UTC]
[05:25:30] + Processing work unit
[05:25:30] Trying to send all finished work units
[05:25:30] Core required: FahCore_a3.exe
[05:25:30] + No unsent completed units remaining.
[05:25:30] Core found.
[05:25:30] - Autosend completed
[05:25:30] Working on queue slot 00 [December 3 05:25:30 UTC]
[05:25:30] + Working ...
[05:25:30] - Calling '.\FahCore_a3.exe -dir work/ -nice 19 -suffix 00 -np 24 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 3636 -version 630'

[05:25:31] 
[05:25:31] *------------------------------*
[05:25:31] Folding@Home Gromacs SMP Core
[05:25:31] Version 2.22 (Mar 12, 2010)
[05:25:31] 
[05:25:31] Preparing to commence simulation
[05:25:31] - Ensuring status. Please wait.
[05:25:40] - Assembly optimizations manually forced on.
[05:25:40] - Not checking prior termination.
[05:25:45] - Expanded 24856761 -> 30796293 (decompressed 123.8 percent)
[05:25:45] Called DecompressByteArray: compressed_data_size=24856761 data_size=30796293, decompressed_data_size=30796293 diff=0
[05:25:45] - Digital signature verified
[05:25:45] 
[05:25:45] Project: 6900 (Run 30, Clone 19, Gen 2)
[05:25:45] 
[05:25:45] Assembly optimizations on if available.
[05:25:45] Entering M.D.
[05:25:51] Using Gromacs checkpoints
[05:26:00] Resuming from checkpoint
[05:26:00] Verified work/wudata_00.log
[05:26:01] Verified work/wudata_00.trr
[05:26:01] Verified work/wudata_00.xtc
[05:26:01] Verified work/wudata_00.edr
[05:26:01] Completed 41840 out of 250000 steps  (16%)
[05:28:54] Completed 42500 out of 250000 steps  (17%)
[05:39:56] Completed 45000 out of 250000 steps  (18%)
[05:50:56] Completed 47500 out of 250000 steps  (19%)
[06:01:01] CoreStatus = C0000005 (-1073741819)
[06:01:01] Client-core communications error: ERROR 0xc0000005
[06:01:01] Deleting current work unit & continuing...
[06:01:17] Trying to send all finished work units
[06:01:17] + No unsent completed units remaining.
[06:01:17] - Preparing to get new work unit...
[06:01:17] Cleaning up work directory
[06:01:17] + Attempting to get work packet
[06:01:17] Passkey found
[06:01:17] - Will indicate memory of 12279 MB
[06:01:17] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 12, Stepping: 2
[06:01:17] - Connecting to assignment server
[06:01:17] Connecting to http://assign.stanford.edu:8080/
[06:01:18] Posted data.
[06:01:18] Initial: 43AB; - Successful: assigned to (171.67.108.22).
[06:01:18] + News From Folding@Home: Welcome to Folding@Home
[06:01:18] Loaded queue successfully.
[06:01:18] Sent data
[06:01:18] Connecting to http://171.67.108.22:8080/
[06:01:27] Posted data.
[06:01:27] Initial: 0000; - Receiving payload (expected size: 25463482)
[06:02:26] - Downloaded at ~421 kB/s
[06:02:26] - Averaged speed for that direction ~307 kB/s
[06:02:26] + Received work.
[06:02:27] + Closed connections
[06:02:32] 
[06:02:32] + Processing work unit
[06:02:32] Core required: FahCore_a3.exe
[06:02:32] Core found.
[06:02:32] Working on queue slot 01 [December 3 06:02:32 UTC]
[06:02:32] + Working ...
[06:02:32] - Calling '.\FahCore_a3.exe -dir work/ -nice 19 -suffix 01 -np 24 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 3636 -version 630'

[06:02:32] 
[06:02:32] *------------------------------*
[06:02:32] Folding@Home Gromacs SMP Core
[06:02:32] Version 2.22 (Mar 12, 2010)
[06:02:32] 
[06:02:32] Preparing to commence simulation
[06:02:32] - Assembly optimizations manually forced on.
[06:02:32] - Not checking prior termination.
[06:02:36] - Expanded 25462970 -> 31941441 (decompressed 125.4 percent)
[06:02:36] Called DecompressByteArray: compressed_data_size=25462970 data_size=31941441, decompressed_data_size=31941441 diff=0
[06:02:37] - Digital signature verified
[06:02:37] 
[06:02:37] Project: 2686 (Run 3, Clone 19, Gen 40)
[06:02:37] 
[06:02:37] Assembly optimizations on if available.
[06:02:37] Entering M.D.
[06:02:45] Completed 0 out of 250000 steps  (0%)

[sortofageek]

Just to confirm, this WU was completed successfully by another folder and the results were added to the database on 2010-12-04 14:08:37 for 121134 points of credit.

[MIBW]

Sounds like it was me then... I presumed as much and made adjustments for stability... thanks for following that up.

[sortofageek]

If you have a log showing you sending home the completed WU about that time, it may have been you. The folding name starts with a D and the team number has two digits. The only way I can confirm or deny is if you give us your folding name and team number.

[MIBW]

Oh sorry, that would make it easier - I fold under "digitalFX" for team 33, the one, the only [H].

Can't find any completed logfile - just the 2 stabs at it mentioned earlier. EDIT - nope, HFM was messing with me - found it

[sortofageek]

Good news. It was you, congrats!

[MIBW]

Ok, I had just missed it in the HFM log - it wraps around - tired eyes.

So they auto assigned me a 2686, in which time I had sorted my problem, and after that finished I got this same 6900 back but this time completed it.

So I did it 1.9 times to be thorough.

[sortofageek]

Thanks for that. We like our protein thoroughly chewed.