Project: 8020 (Run 4, Clone 222, Gen 80) - NaN

[Fahrenheit451]

Project: 8020 (Run 4, Clone 222, Gen 80) quits @ 59% with a "mdrun_gpu returned 52". It was the second WU on my new GTX 560 Ti replacement. Here is the log:

Code: Select all

*********************** Log Started 2012-06-23T08:55:32Z ***********************
08:55:32:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
08:55:33:WU00:FS00:News: Welcome to Folding@Home
08:55:33:WU00:FS00:Assigned to work server 171.67.108.143
08:55:33:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.67.108.143
08:55:33:WU00:FS00:Connecting to 171.67.108.143:8080
08:55:33:WU00:FS00:Downloading 142.91KiB
08:55:35:WU00:FS00:Download complete
08:55:35:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:OK project:8020 run:1 clone:145 gen:31 core:0x15 unit:0x0000002c6953ee2f4f96790436da2877
08:55:35:WU00:FS00:Starting
08:55:35:WU00:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
08:55:35:WU00:FS00:Started FahCore on PID 3532
08:55:35:WU00:FS00:Core PID:940
08:55:35:WU00:FS00:FahCore 0x15 started
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:*------------------------------*
08:55:36:WU00:FS00:0x15:Folding@Home GPU Core
08:55:36:WU00:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
08:55:36:WU00:FS00:0x15:Build host             SimbiosNvdWin7
08:55:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
08:55:36:WU00:FS00:0x15:Core                   15
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Window's signal control handler registered.
08:55:36:WU00:FS00:0x15:Preparing to commence simulation
08:55:36:WU00:FS00:0x15:- Looking at optimizations...
08:55:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
08:55:36:WU00:FS00:0x15:- Created dyn
08:55:36:WU00:FS00:0x15:- Files status OK
08:55:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
08:55:36:WU00:FS00:0x15:- Expanded 145832 -> 660994 (decompressed 453.2 percent)
08:55:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=145832 data_size=660994, decompressed_data_size=660994 diff=0
08:55:36:WU00:FS00:0x15:- Digital signature verified
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Project: 8020 (Run 1, Clone 145, Gen 31)
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Assembly optimizations on if available.
08:55:36:WU00:FS00:0x15:Entering M.D.
08:55:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2354779254 2686148474 3849180432 2176399459 508864011
08:55:37:WU00:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
08:55:37:WU00:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
08:55:37:WU00:FS00:0x15:Client config unavailable.
08:55:38:WU00:FS00:0x15:Starting GUI Server
08:56:40:WU00:FS00:0x15:Setting checkpoint frequency: 250000
08:56:41:WU00:FS00:0x15:Completed         3 out of 25000000 steps (0%).
09:00:53:WU00:FS00:0x15:Completed    250000 out of 25000000 steps (1%).
09:05:07:WU00:FS00:0x15:Completed    500000 out of 25000000 steps (2%).
09:09:21:WU00:FS00:0x15:Completed    750000 out of 25000000 steps (3%).
09:13:34:WU00:FS00:0x15:Completed   1000000 out of 25000000 steps (4%).
09:17:47:WU00:FS00:0x15:Completed   1250000 out of 25000000 steps (5%).
09:22:01:WU00:FS00:0x15:Completed   1500000 out of 25000000 steps (6%).
09:26:15:WU00:FS00:0x15:Completed   1750000 out of 25000000 steps (7%).
09:30:28:WU00:FS00:0x15:Completed   2000000 out of 25000000 steps (8%).
09:34:41:WU00:FS00:0x15:Completed   2250000 out of 25000000 steps (9%).
09:38:54:WU00:FS00:0x15:Completed   2500000 out of 25000000 steps (10%).
09:43:08:WU00:FS00:0x15:Completed   2750000 out of 25000000 steps (11%).
09:47:21:WU00:FS00:0x15:Completed   3000000 out of 25000000 steps (12%).
09:51:35:WU00:FS00:0x15:Completed   3250000 out of 25000000 steps (13%).
09:55:48:WU00:FS00:0x15:Completed   3500000 out of 25000000 steps (14%).
10:00:01:WU00:FS00:0x15:Completed   3750000 out of 25000000 steps (15%).
10:04:15:WU00:FS00:0x15:Completed   4000000 out of 25000000 steps (16%).
10:08:28:WU00:FS00:0x15:Completed   4250000 out of 25000000 steps (17%).
10:12:41:WU00:FS00:0x15:Completed   4500000 out of 25000000 steps (18%).
10:16:55:WU00:FS00:0x15:Completed   4750000 out of 25000000 steps (19%).
10:21:08:WU00:FS00:0x15:Completed   5000000 out of 25000000 steps (20%).
10:25:21:WU00:FS00:0x15:Completed   5250000 out of 25000000 steps (21%).
10:29:35:WU00:FS00:0x15:Completed   5500000 out of 25000000 steps (22%).
10:33:48:WU00:FS00:0x15:Completed   5750000 out of 25000000 steps (23%).
10:38:01:WU00:FS00:0x15:Completed   6000000 out of 25000000 steps (24%).
10:42:15:WU00:FS00:0x15:Completed   6250000 out of 25000000 steps (25%).
10:46:28:WU00:FS00:0x15:Completed   6500000 out of 25000000 steps (26%).
10:50:41:WU00:FS00:0x15:Completed   6750000 out of 25000000 steps (27%).
10:54:55:WU00:FS00:0x15:Completed   7000000 out of 25000000 steps (28%).
10:59:08:WU00:FS00:0x15:Completed   7250000 out of 25000000 steps (29%).
11:03:21:WU00:FS00:0x15:Completed   7500000 out of 25000000 steps (30%).
11:07:35:WU00:FS00:0x15:Completed   7750000 out of 25000000 steps (31%).
11:11:48:WU00:FS00:0x15:Completed   8000000 out of 25000000 steps (32%).
11:16:02:WU00:FS00:0x15:Completed   8250000 out of 25000000 steps (33%).
11:20:15:WU00:FS00:0x15:Completed   8500000 out of 25000000 steps (34%).
11:24:28:WU00:FS00:0x15:Completed   8750000 out of 25000000 steps (35%).
11:28:41:WU00:FS00:0x15:Completed   9000000 out of 25000000 steps (36%).
11:32:55:WU00:FS00:0x15:Completed   9250000 out of 25000000 steps (37%).
11:37:08:WU00:FS00:0x15:Completed   9500000 out of 25000000 steps (38%).
11:41:22:WU00:FS00:0x15:Completed   9750000 out of 25000000 steps (39%).
11:45:35:WU00:FS00:0x15:Completed  10000000 out of 25000000 steps (40%).
11:49:48:WU00:FS00:0x15:Completed  10250000 out of 25000000 steps (41%).
11:54:01:WU00:FS00:0x15:Completed  10500000 out of 25000000 steps (42%).
11:58:15:WU00:FS00:0x15:Completed  10750000 out of 25000000 steps (43%).
12:02:28:WU00:FS00:0x15:Completed  11000000 out of 25000000 steps (44%).
12:06:42:WU00:FS00:0x15:Completed  11250000 out of 25000000 steps (45%).
12:10:55:WU00:FS00:0x15:Completed  11500000 out of 25000000 steps (46%).
12:15:08:WU00:FS00:0x15:Completed  11750000 out of 25000000 steps (47%).
12:19:21:WU00:FS00:0x15:Completed  12000000 out of 25000000 steps (48%).
12:23:35:WU00:FS00:0x15:Completed  12250000 out of 25000000 steps (49%).
12:27:48:WU00:FS00:0x15:Completed  12500000 out of 25000000 steps (50%).
12:32:02:WU00:FS00:0x15:Completed  12750000 out of 25000000 steps (51%).
12:36:15:WU00:FS00:0x15:Completed  13000000 out of 25000000 steps (52%).
12:40:28:WU00:FS00:0x15:Completed  13250000 out of 25000000 steps (53%).
12:44:42:WU00:FS00:0x15:Completed  13500000 out of 25000000 steps (54%).
12:48:55:WU00:FS00:0x15:Completed  13750000 out of 25000000 steps (55%).
12:53:08:WU00:FS00:0x15:Completed  14000000 out of 25000000 steps (56%).
12:57:22:WU00:FS00:0x15:Completed  14250000 out of 25000000 steps (57%).
13:01:35:WU00:FS00:0x15:Completed  14500000 out of 25000000 steps (58%).
13:05:48:WU00:FS00:0x15:Completed  14750000 out of 25000000 steps (59%).
13:10:02:WU00:FS00:0x15:Completed  15000000 out of 25000000 steps (60%).
13:14:15:WU00:FS00:0x15:Completed  15250000 out of 25000000 steps (61%).
13:18:28:WU00:FS00:0x15:Completed  15500000 out of 25000000 steps (62%).
13:22:42:WU00:FS00:0x15:Completed  15750000 out of 25000000 steps (63%).
13:26:55:WU00:FS00:0x15:Completed  16000000 out of 25000000 steps (64%).
13:31:09:WU00:FS00:0x15:Completed  16250000 out of 25000000 steps (65%).
13:35:22:WU00:FS00:0x15:Completed  16500000 out of 25000000 steps (66%).
13:39:35:WU00:FS00:0x15:Completed  16750000 out of 25000000 steps (67%).
13:43:49:WU00:FS00:0x15:Completed  17000000 out of 25000000 steps (68%).
13:48:02:WU00:FS00:0x15:Completed  17250000 out of 25000000 steps (69%).
13:52:15:WU00:FS00:0x15:Completed  17500000 out of 25000000 steps (70%).
13:56:29:WU00:FS00:0x15:Completed  17750000 out of 25000000 steps (71%).
14:00:42:WU00:FS00:0x15:Completed  18000000 out of 25000000 steps (72%).
14:04:55:WU00:FS00:0x15:Completed  18250000 out of 25000000 steps (73%).
14:09:09:WU00:FS00:0x15:Completed  18500000 out of 25000000 steps (74%).
14:13:22:WU00:FS00:0x15:Completed  18750000 out of 25000000 steps (75%).
14:17:35:WU00:FS00:0x15:Completed  19000000 out of 25000000 steps (76%).
14:21:49:WU00:FS00:0x15:Completed  19250000 out of 25000000 steps (77%).
14:26:02:WU00:FS00:0x15:Completed  19500000 out of 25000000 steps (78%).
14:30:15:WU00:FS00:0x15:Completed  19750000 out of 25000000 steps (79%).
14:34:29:WU00:FS00:0x15:Completed  20000000 out of 25000000 steps (80%).
14:38:42:WU00:FS00:0x15:Completed  20250000 out of 25000000 steps (81%).
14:42:55:WU00:FS00:0x15:Completed  20500000 out of 25000000 steps (82%).
14:47:09:WU00:FS00:0x15:Completed  20750000 out of 25000000 steps (83%).
14:51:22:WU00:FS00:0x15:Completed  21000000 out of 25000000 steps (84%).
******************************** Date: 23/06/12 ********************************
14:55:36:WU00:FS00:0x15:Completed  21250000 out of 25000000 steps (85%).
14:59:49:WU00:FS00:0x15:Completed  21500000 out of 25000000 steps (86%).
15:04:02:WU00:FS00:0x15:Completed  21750000 out of 25000000 steps (87%).
15:08:16:WU00:FS00:0x15:Completed  22000000 out of 25000000 steps (88%).
15:12:29:WU00:FS00:0x15:Completed  22250000 out of 25000000 steps (89%).
15:16:42:WU00:FS00:0x15:Completed  22500000 out of 25000000 steps (90%).
15:20:56:WU00:FS00:0x15:Completed  22750000 out of 25000000 steps (91%).
15:25:09:WU00:FS00:0x15:Completed  23000000 out of 25000000 steps (92%).
15:29:22:WU00:FS00:0x15:Completed  23250000 out of 25000000 steps (93%).
15:33:36:WU00:FS00:0x15:Completed  23500000 out of 25000000 steps (94%).
15:37:49:WU00:FS00:0x15:Completed  23750000 out of 25000000 steps (95%).
15:42:02:WU00:FS00:0x15:Completed  24000000 out of 25000000 steps (96%).
15:46:16:WU00:FS00:0x15:Completed  24250000 out of 25000000 steps (97%).
15:50:29:WU00:FS00:0x15:Completed  24500000 out of 25000000 steps (98%).
15:54:42:WU00:FS00:0x15:Completed  24750000 out of 25000000 steps (99%).
15:54:43:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
15:54:44:WU02:FS00:News: Welcome to Folding@Home
15:54:44:WU02:FS00:Assigned to work server 171.67.108.143
15:54:44:WU02:FS00:Requesting new work unit for slot 00: RUNNING gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.67.108.143
15:54:44:WU02:FS00:Connecting to 171.67.108.143:8080
15:54:45:WU02:FS00:Downloading 142.81KiB
15:54:47:WU02:FS00:Download complete
15:54:47:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:OK project:8020 run:4 clone:222 gen:80 core:0x15 unit:0x000000696953ee2f4f9679fdc4b0fe66
15:58:56:WU00:FS00:0x15:Completed  25000000 out of 25000000 steps (100%).
15:58:57:WU00:FS00:0x15:Finished fah_main status=0
15:58:57:WU00:FS00:0x15:Successful run
15:58:57:WU00:FS00:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
15:59:07:WU00:FS00:0x15:Reserved 436428 bytes for xtc file; Cosm status=0
15:59:07:WU00:FS00:0x15:Allocated 436428 bytes for xtc file
15:59:07:WU00:FS00:0x15:- Reading up to 436428 from "00/wudata_01.xtc": Read 436428
15:59:07:WU00:FS00:0x15:Read 436428 bytes from xtc file; available packet space=785994036
15:59:07:WU00:FS00:0x15:xtc file hash check passed.
15:59:07:WU00:FS00:0x15:Reserved 54408 54408 785994036 bytes for arc file=<00/wudata_01.trr> Cosm status=0
15:59:07:WU00:FS00:0x15:Allocated 54408 bytes for arc file
15:59:07:WU00:FS00:0x15:- Reading up to 54408 from "00/wudata_01.trr": Read 54408
15:59:07:WU00:FS00:0x15:Read 54408 bytes from arc file; available packet space=785939628
15:59:07:WU00:FS00:0x15:trr file hash check passed.
15:59:07:WU00:FS00:0x15:Allocated 544 bytes for edr file
15:59:07:WU00:FS00:0x15:Read bedfile
15:59:07:WU00:FS00:0x15:edr file hash check passed.
15:59:07:WU00:FS00:0x15:Allocated 32222 bytes for logfile
15:59:07:WU00:FS00:0x15:Read logfile
15:59:07:WU00:FS00:0x15:GuardedRun: success in DynamicWrapper
15:59:07:WU00:FS00:0x15:GuardedRun: done
15:59:07:WU00:FS00:0x15:Run: GuardedRun completed.
15:59:07:WU00:FS00:0x15:+ Opened results file
15:59:07:WU00:FS00:0x15:- Writing 524114 bytes of core data to disk...
15:59:07:WU00:FS00:0x15:Done: 523602 -> 501158 (compressed to 95.7 percent)
15:59:07:WU00:FS00:0x15:  ... Done.
15:59:07:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
15:59:07:WU00:FS00:0x15:Shutting down core 
15:59:07:WU00:FS00:0x15:
15:59:07:WU00:FS00:0x15:Folding@home Core Shutdown: FINISHED_UNIT
15:59:07:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
15:59:07:WU00:FS00:Sending unit results: id:00 state:SEND error:OK project:8020 run:1 clone:145 gen:31 core:0x15 unit:0x0000002c6953ee2f4f96790436da2877
15:59:07:WU00:FS00:Uploading 489.91KiB to 171.67.108.143
15:59:07:WU02:FS00:Starting
15:59:07:WU00:FS00:Connecting to 171.67.108.143:8080
15:59:07:WU02:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
15:59:07:WU02:FS00:Started FahCore on PID 3900
15:59:07:WU02:FS00:Core PID:3044
15:59:07:WU02:FS00:FahCore 0x15 started
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:*------------------------------*
15:59:08:WU02:FS00:0x15:Folding@Home GPU Core
15:59:08:WU02:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
15:59:08:WU02:FS00:0x15:Build host             SimbiosNvdWin7
15:59:08:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
15:59:08:WU02:FS00:0x15:Core                   15
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Window's signal control handler registered.
15:59:08:WU02:FS00:0x15:Preparing to commence simulation
15:59:08:WU02:FS00:0x15:- Looking at optimizations...
15:59:08:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
15:59:08:WU02:FS00:0x15:- Created dyn
15:59:08:WU02:FS00:0x15:- Files status OK
15:59:08:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:59:08:WU02:FS00:0x15:- Expanded 145726 -> 660994 (decompressed 453.5 percent)
15:59:08:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=145726 data_size=660994, decompressed_data_size=660994 diff=0
15:59:08:WU02:FS00:0x15:- Digital signature verified
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Project: 8020 (Run 4, Clone 222, Gen 80)
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Assembly optimizations on if available.
15:59:08:WU02:FS00:0x15:Entering M.D.
15:59:10:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1984467589 290667450 1345239047 429439430 2573288279
15:59:10:WU02:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
15:59:10:WU02:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
15:59:10:WU02:FS00:0x15:Client config unavailable.
15:59:10:WU02:FS00:0x15:Starting GUI Server
15:59:13:WU00:FS00:Upload 65.32%
15:59:16:WU00:FS00:Upload complete
15:59:16:WU00:FS00:Server responded WORK_ACK (400)
15:59:16:WU00:FS00:Final credit estimate, 5757.00 points
15:59:16:WU00:FS00:Cleaning up
16:00:11:WU02:FS00:0x15:Setting checkpoint frequency: 250000
16:00:11:WU02:FS00:0x15:Completed         3 out of 25000000 steps (0%).
16:04:24:WU02:FS00:0x15:Completed    250000 out of 25000000 steps (1%).
16:08:37:WU02:FS00:0x15:Completed    500000 out of 25000000 steps (2%).
16:12:51:WU02:FS00:0x15:Completed    750000 out of 25000000 steps (3%).
16:17:04:WU02:FS00:0x15:Completed   1000000 out of 25000000 steps (4%).
16:21:17:WU02:FS00:0x15:Completed   1250000 out of 25000000 steps (5%).
16:25:31:WU02:FS00:0x15:Completed   1500000 out of 25000000 steps (6%).
16:29:44:WU02:FS00:0x15:Completed   1750000 out of 25000000 steps (7%).
16:33:57:WU02:FS00:0x15:Completed   2000000 out of 25000000 steps (8%).
16:38:11:WU02:FS00:0x15:Completed   2250000 out of 25000000 steps (9%).
16:42:24:WU02:FS00:0x15:Completed   2500000 out of 25000000 steps (10%).
16:46:38:WU02:FS00:0x15:Completed   2750000 out of 25000000 steps (11%).
16:50:51:WU02:FS00:0x15:Completed   3000000 out of 25000000 steps (12%).
16:55:04:WU02:FS00:0x15:Completed   3250000 out of 25000000 steps (13%).
16:59:17:WU02:FS00:0x15:Completed   3500000 out of 25000000 steps (14%).
17:03:31:WU02:FS00:0x15:Completed   3750000 out of 25000000 steps (15%).
17:07:44:WU02:FS00:0x15:Completed   4000000 out of 25000000 steps (16%).
17:11:58:WU02:FS00:0x15:Completed   4250000 out of 25000000 steps (17%).
17:16:11:WU02:FS00:0x15:Completed   4500000 out of 25000000 steps (18%).
17:20:24:WU02:FS00:0x15:Completed   4750000 out of 25000000 steps (19%).
17:24:38:WU02:FS00:0x15:Completed   5000000 out of 25000000 steps (20%).
17:28:51:WU02:FS00:0x15:Completed   5250000 out of 25000000 steps (21%).
17:33:05:WU02:FS00:0x15:Completed   5500000 out of 25000000 steps (22%).
17:37:18:WU02:FS00:0x15:Completed   5750000 out of 25000000 steps (23%).
17:41:31:WU02:FS00:0x15:Completed   6000000 out of 25000000 steps (24%).
17:45:45:WU02:FS00:0x15:Completed   6250000 out of 25000000 steps (25%).
17:49:58:WU02:FS00:0x15:Completed   6500000 out of 25000000 steps (26%).
17:54:11:WU02:FS00:0x15:Completed   6750000 out of 25000000 steps (27%).
17:58:24:WU02:FS00:0x15:Completed   7000000 out of 25000000 steps (28%).
18:02:38:WU02:FS00:0x15:Completed   7250000 out of 25000000 steps (29%).
18:06:51:WU02:FS00:0x15:Completed   7500000 out of 25000000 steps (30%).
18:11:05:WU02:FS00:0x15:Completed   7750000 out of 25000000 steps (31%).
18:15:18:WU02:FS00:0x15:Completed   8000000 out of 25000000 steps (32%).
18:16:05:FS00:Finishing
18:19:32:WU02:FS00:0x15:Completed   8250000 out of 25000000 steps (33%).
18:23:45:WU02:FS00:0x15:Completed   8500000 out of 25000000 steps (34%).
18:27:59:WU02:FS00:0x15:Completed   8750000 out of 25000000 steps (35%).
18:32:13:WU02:FS00:0x15:Completed   9000000 out of 25000000 steps (36%).
18:36:26:WU02:FS00:0x15:Completed   9250000 out of 25000000 steps (37%).
18:40:40:WU02:FS00:0x15:Completed   9500000 out of 25000000 steps (38%).
18:44:54:WU02:FS00:0x15:Completed   9750000 out of 25000000 steps (39%).
18:49:07:WU02:FS00:0x15:Completed  10000000 out of 25000000 steps (40%).
18:53:21:WU02:FS00:0x15:Completed  10250000 out of 25000000 steps (41%).
18:57:35:WU02:FS00:0x15:Completed  10500000 out of 25000000 steps (42%).
19:01:48:WU02:FS00:0x15:Completed  10750000 out of 25000000 steps (43%).
19:06:02:WU02:FS00:0x15:Completed  11000000 out of 25000000 steps (44%).
19:10:16:WU02:FS00:0x15:Completed  11250000 out of 25000000 steps (45%).
19:14:30:WU02:FS00:0x15:Completed  11500000 out of 25000000 steps (46%).
19:18:43:WU02:FS00:0x15:Completed  11750000 out of 25000000 steps (47%).
19:22:57:WU02:FS00:0x15:Completed  12000000 out of 25000000 steps (48%).
19:27:10:WU02:FS00:0x15:Completed  12250000 out of 25000000 steps (49%).
19:31:24:WU02:FS00:0x15:Completed  12500000 out of 25000000 steps (50%).
19:35:38:WU02:FS00:0x15:Completed  12750000 out of 25000000 steps (51%).
19:39:51:WU02:FS00:0x15:Completed  13000000 out of 25000000 steps (52%).
19:44:05:WU02:FS00:0x15:Completed  13250000 out of 25000000 steps (53%).
19:48:19:WU02:FS00:0x15:Completed  13500000 out of 25000000 steps (54%).
19:52:32:WU02:FS00:0x15:Completed  13750000 out of 25000000 steps (55%).
19:56:46:WU02:FS00:0x15:Completed  14000000 out of 25000000 steps (56%).
20:01:00:WU02:FS00:0x15:Completed  14250000 out of 25000000 steps (57%).
20:05:13:WU02:FS00:0x15:Completed  14500000 out of 25000000 steps (58%).
20:09:27:WU02:FS00:0x15:Completed  14750000 out of 25000000 steps (59%).
20:09:27:WU02:FS00:0x15:mdrun_gpu returned 52
20:09:27:WU02:FS00:0x15:NANs detected on GPU
20:09:27:WU02:FS00:0x15:
20:09:27:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
20:09:28:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
20:09:28:WU02:FS00:Starting
20:09:28:WU02:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
20:09:28:WU02:FS00:Started FahCore on PID 4192
20:09:28:WU02:FS00:Core PID:1100
20:09:28:WU02:FS00:FahCore 0x15 started
20:09:28:WU02:FS00:0x15:
20:09:28:WU02:FS00:0x15:*------------------------------*
20:09:28:WU02:FS00:0x15:Folding@Home GPU Core
20:09:28:WU02:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
20:09:28:WU02:FS00:0x15:Build host             SimbiosNvdWin7
20:09:28:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
20:09:28:WU02:FS00:0x15:Core                   15
20:09:28:WU02:FS00:0x15:
20:09:28:WU02:FS00:0x15:Window's signal control handler registered.
20:09:28:WU02:FS00:0x15:Preparing to commence simulation
20:09:28:WU02:FS00:0x15:- Looking at optimizations...
20:09:28:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
20:09:28:WU02:FS00:0x15:- Created dyn
20:09:28:WU02:FS00:0x15:- Files status OK
20:09:28:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
20:09:28:WU02:FS00:0x15:- Expanded 145726 -> 660994 (decompressed 453.5 percent)
20:09:28:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=145726 data_size=660994, decompressed_data_size=660994 diff=0
20:09:29:WU02:FS00:0x15:- Digital signature verified
20:09:29:WU02:FS00:0x15:
20:09:29:WU02:FS00:0x15:Project: 8020 (Run 4, Clone 222, Gen 80)
20:09:29:WU02:FS00:0x15:
20:09:29:WU02:FS00:0x15:Assembly optimizations on if available.
20:09:29:WU02:FS00:0x15:Entering M.D.
20:09:31:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1984467589 290667450 1345239047 429439430 2573288279
20:09:31:WU02:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
20:09:31:WU02:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
20:09:31:WU02:FS00:0x15:Client config unavailable.
20:09:31:WU02:FS00:0x15:Starting GUI Server
20:10:33:WARNING:WU02:FS00:Detected clock skew, adjusting time estimates
20:10:33:WU02:FS00:0x15:Setting checkpoint frequency: 250000
20:10:33:WU02:FS00:0x15:Completed         3 out of 25000000 steps (0%).
20:14:45:WU02:FS00:0x15:Completed    250000 out of 25000000 steps (1%).
20:18:59:WU02:FS00:0x15:Completed    500000 out of 25000000 steps (2%).
20:23:13:WU02:FS00:0x15:Completed    750000 out of 25000000 steps (3%).

[PantherX]

No data is available in the WU Database for Project: 8020 (Run 4, Clone 222, Gen 80) so I have marked it for a follow-up.

[Fahrenheit451]

I got the same WU assigned twice after my report. In the second case it crashed with NaN @ 42% and in third case I quit the client. See log file below. Please apologize that it is a bit hard to read due to the SMP/GPU mix.

Code: Select all

*********************** Log Started 2012-06-23T08:55:32Z ***********************
08:55:32:************************* Folding@home Client *************************
08:55:32:      Website: http://folding.stanford.edu/
08:55:32:    Copyright: (c) 2009-2012 Stanford University
08:55:32:       Author: Joseph Coffland <[email protected]>
08:55:32:         Args: --lifeline 3312 --command-port=36330
08:55:32:       Config: C:/ProgramData/FAHClient/config.xml
08:55:32:******************************** Build ********************************
08:55:32:      Version: 7.1.52
08:55:32:         Date: Mar 20 2012
08:55:32:         Time: 19:37:42
08:55:32:      SVN Rev: 3515
08:55:32:       Branch: fah/trunk/client
08:55:32:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
08:55:32:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
08:55:32:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT
08:55:32:     Platform: win32 XP
08:55:32:         Bits: 32
08:55:32:         Mode: Release
08:55:32:******************************* System ********************************
08:55:32:          CPU: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
08:55:32:       CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
08:55:32:         CPUs: 8
08:55:32:       Memory: 7.98GiB
08:55:32:  Free Memory: 6.73GiB
08:55:32:      Threads: WINDOWS_THREADS
08:55:32:   On Battery: false
08:55:32:   UTC offset: 2
08:55:32:          PID: 1740
08:55:32:          CWD: C:/ProgramData/FAHClient
08:55:32:           OS: Windows 7 Ultimate
08:55:32:      OS Arch: AMD64
08:55:32:         GPUs: 1
08:55:32:        GPU 0: FERMI:1 GF114 [GeForce GTX 560 Ti]
08:55:32:         CUDA: 2.1
08:55:32:  CUDA Driver: 4020
08:55:32:Win32 Service: false
08:55:32:***********************************************************************
08:55:32:<config>
08:55:32:  <service-description v='Folding@home Client'/>
08:55:32:  <service-restart v='true'/>
08:55:32:  <service-restart-delay v='5000'/>
08:55:32:
08:55:32:  <!-- Client Control -->
08:55:32:  <cycle-rate v='4'/>
08:55:32:  <cycles v='-1'/>
08:55:32:  <data-directory v='.'/>
08:55:32:  <disable-project-lookup v='false'/>
08:55:32:  <exec-directory v='D:\FAHClient_V7'/>
08:55:32:  <exit-when-done v='false'/>
08:55:32:  <threads v='4'/>
08:55:32:
08:55:32:  <!-- Configuration -->
08:55:32:  <config-rotate v='true'/>
08:55:32:  <config-rotate-dir v='configs'/>
08:55:32:  <config-rotate-max v='16'/>
08:55:32:
08:55:32:  <!-- Debugging -->
08:55:32:  <assignment-servers>
08:55:32:    assign3.stanford.edu:8080 assign4.stanford.edu:80
08:55:32:  </assignment-servers>
08:55:32:  <capture-directory v='capture'/>
08:55:32:  <capture-sockets v='false'/>
08:55:32:  <debug-sockets v='false'/>
08:55:32:  <exception-locations v='true'/>
08:55:32:  <gpu-assignment-servers>
08:55:32:    assign-GPU.stanford.edu:80 assign-GPU.stanford.edu:8080
08:55:32:  </gpu-assignment-servers>
08:55:32:  <stack-traces v='false'/>
08:55:32:
08:55:32:  <!-- Error Handling -->
08:55:32:  <max-slot-errors v='5'/>
08:55:32:  <max-unit-errors v='5'/>
08:55:32:
08:55:32:  <!-- FahCore Control -->
08:55:32:  <checkpoint v='15'/>
08:55:32:  <core-dir v='cores'/>
08:55:32:  <core-priority v='idle'/>
08:55:32:  <cpu-affinity v='false'/>
08:55:32:  <cpu-usage v='100'/>
08:55:32:  <no-assembly v='false'/>
08:55:32:
08:55:32:  <!-- Folding Slot Configuration -->
08:55:32:  <client-subtype v='STDCLI'/>
08:55:32:  <client-type v='advanced'/>
08:55:32:  <cpu-species v='X86_PENTIUM_II'/>
08:55:32:  <cpu-type v='AMD64'/>
08:55:32:  <cpus v='-1'/>
08:55:32:  <cuda-index v='0'/>
08:55:32:  <gpu v='true'/>
08:55:32:  <gpu-usage v='100'/>
08:55:32:  <max-packet-size v='big'/>
08:55:32:  <opencl-index v='0'/>
08:55:32:  <os-species v='UNKNOWN'/>
08:55:32:  <os-type v='WIN32'/>
08:55:32:  <project-key v='0'/>
08:55:32:  <smp v='true'/>
08:55:32:
08:55:32:  <!-- Logging -->
08:55:32:  <log v='log.txt'/>
08:55:32:  <log-color v='false'/>
08:55:32:  <log-crlf v='true'/>
08:55:32:  <log-date v='false'/>
08:55:32:  <log-date-periodically v='21600'/>
08:55:32:  <log-debug v='true'/>
08:55:32:  <log-domain v='false'/>
08:55:32:  <log-header v='true'/>
08:55:32:  <log-level v='true'/>
08:55:32:  <log-no-info-header v='true'/>
08:55:32:  <log-redirect v='false'/>
08:55:32:  <log-rotate v='true'/>
08:55:32:  <log-rotate-dir v='logs'/>
08:55:32:  <log-rotate-max v='16'/>
08:55:32:  <log-short-level v='false'/>
08:55:32:  <log-simple-domains v='true'/>
08:55:32:  <log-thread-id v='false'/>
08:55:32:  <log-thread-prefix v='true'/>
08:55:32:  <log-time v='true'/>
08:55:32:  <log-to-screen v='true'/>
08:55:32:  <log-truncate v='false'/>
08:55:32:  <verbosity v='5'/>
08:55:32:
08:55:32:  <!-- Network -->
08:55:32:  <proxy v=':8080'/>
08:55:32:  <proxy-enable v='false'/>
08:55:32:  <proxy-pass v=''/>
08:55:32:  <proxy-user v=''/>
08:55:32:
08:55:32:  <!-- Process Control -->
08:55:32:  <child v='false'/>
08:55:32:  <daemon v='false'/>
08:55:32:  <pid v='false'/>
08:55:32:  <pid-file v='Folding@home Client.pid'/>
08:55:32:  <respawn v='false'/>
08:55:32:  <service v='false'/>
08:55:32:
08:55:32:  <!-- Remote Command Server -->
08:55:32:  <command-address v='0.0.0.0'/>
08:55:32:  <command-allow v='127.0.0.1,192.168.2.0/24'/>
08:55:32:  <command-allow-no-pass v='127.0.0.1,192.168.2.0/24'/>
08:55:32:  <command-deny v='0.0.0.0/0'/>
08:55:32:  <command-deny-no-pass v='0.0.0.0/0'/>
08:55:32:  <command-port v='36330'/>
08:55:32:
08:55:32:  <!-- Slot Control -->
08:55:32:  <max-shutdown-wait v='60'/>
08:55:32:  <pause-on-battery v='false'/>
08:55:32:  <pause-on-start v='false'/>
08:55:32:
08:55:32:  <!-- User Information -->
08:55:32:  <machine-id v='0'/>
08:55:32:  <passkey v='********************************'/>
08:55:32:  <team v='0'/>
08:55:32:  <user v='superduper4711'/>
08:55:32:
08:55:32:  <!-- Work Unit Control -->
08:55:32:  <dump-after-deadline v='true'/>
08:55:32:  <max-queue v='16'/>
08:55:32:  <max-units v='0'/>
08:55:32:  <next-unit-percentage v='99'/>
08:55:32:
08:55:32:  <!-- Folding Slots -->
08:55:32:  <slot id='0' type='GPU'/>
08:55:32:  <slot id='1' type='SMP'>
08:55:32:    <client-type v='advanced'/>
08:55:32:    <cpus v='-1'/>
08:55:32:    <max-packet-size v='big'/>
08:55:32:  </slot>
08:55:32:</config>
08:55:32:Trying to access database...
08:55:32:Successfully acquired database lock
08:55:32:Enabled folding slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]"
08:55:32:Enabled folding slot 01: READY smp:8
08:55:32:Started thread 4 on PID 1740
08:55:32:Started thread 3 on PID 1740
08:55:32:Started thread 1 on PID 1740
08:55:32:Started thread 6 on PID 1740
08:55:32:Started thread 5 on PID 1740
08:55:32:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
08:55:32:WU01:FS01:Connecting to assign3.stanford.edu:8080
08:55:32:WU01:FS01:News: Welcome to Folding@Home
08:55:32:WU01:FS01:Assigned to work server 171.67.108.60
08:55:32:WU01:FS01:Requesting new work unit for slot 01: READY smp:8 from 171.67.108.60
08:55:32:WU01:FS01:Connecting to 171.67.108.60:8080
08:55:33:WU00:FS00:News: Welcome to Folding@Home
08:55:33:WU00:FS00:Assigned to work server 171.67.108.143
08:55:33:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.67.108.143
08:55:33:WU00:FS00:Connecting to 171.67.108.143:8080
08:55:33:WU00:FS00:Downloading 142.91KiB
08:55:33:WU01:FS01:Downloading 468.92KiB
08:55:34:Server connection id=1 on 0.0.0.0:36330 from 127.0.0.1
08:55:34:Started thread 7 on PID 1740
08:55:35:WU00:FS00:Download complete
08:55:35:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:OK project:8020 run:1 clone:145 gen:31 core:0x15 unit:0x0000002c6953ee2f4f96790436da2877
08:55:35:WU00:FS00:Starting
08:55:35:WU00:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
08:55:35:WU00:FS00:Started FahCore on PID 3532
08:55:35:Started thread 8 on PID 1740
08:55:35:WU00:FS00:Core PID:940
08:55:35:WU00:FS00:FahCore 0x15 started
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:*------------------------------*
08:55:36:WU00:FS00:0x15:Folding@Home GPU Core
08:55:36:WU00:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
08:55:36:WU00:FS00:0x15:Build host             SimbiosNvdWin7
08:55:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
08:55:36:WU00:FS00:0x15:Core                   15
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Window's signal control handler registered.
08:55:36:WU00:FS00:0x15:Preparing to commence simulation
08:55:36:WU00:FS00:0x15:- Looking at optimizations...
08:55:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
08:55:36:WU00:FS00:0x15:- Created dyn
08:55:36:WU00:FS00:0x15:- Files status OK
08:55:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
08:55:36:WU00:FS00:0x15:- Expanded 145832 -> 660994 (decompressed 453.2 percent)
08:55:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=145832 data_size=660994, decompressed_data_size=660994 diff=0
08:55:36:WU00:FS00:0x15:- Digital signature verified
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Project: 8020 (Run 1, Clone 145, Gen 31)
08:55:36:WU00:FS00:0x15:
08:55:36:WU00:FS00:0x15:Assembly optimizations on if available.
08:55:36:WU00:FS00:0x15:Entering M.D.
08:55:36:WU01:FS01:Download complete
08:55:36:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:OK project:8046 run:211 clone:43 gen:21 core:0xa4 unit:0x0000001c6652edcc4fd940e5c1c72e65
08:55:36:WU01:FS01:Starting
08:55:36:WU01:FS01:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -np 8
08:55:36:WU01:FS01:Started FahCore on PID 3172
08:55:36:Started thread 9 on PID 1740
08:55:36:WU01:FS01:Core PID:1356
08:55:36:WU01:FS01:FahCore 0xa4 started
08:55:36:WU01:FS01:0xa4:
08:55:36:WU01:FS01:0xa4:*------------------------------*
08:55:36:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
08:55:36:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
08:55:36:WU01:FS01:0xa4:
08:55:36:WU01:FS01:0xa4:Preparing to commence simulation
08:55:36:WU01:FS01:0xa4:- Looking at optimizations...
08:55:36:WU01:FS01:0xa4:- Created dyn
08:55:36:WU01:FS01:0xa4:- Files status OK
08:55:36:WU01:FS01:0xa4:- Expanded 479664 -> 1077788 (decompressed 224.6 percent)
08:55:36:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=479664 data_size=1077788, decompressed_data_size=1077788 diff=0
08:55:36:WU01:FS01:0xa4:- Digital signature verified
08:55:36:WU01:FS01:0xa4:
08:55:36:WU01:FS01:0xa4:Project: 8046 (Run 211, Clone 43, Gen 21)
08:55:36:WU01:FS01:0xa4:
08:55:36:WU01:FS01:0xa4:Assembly optimizations on if available.
08:55:36:WU01:FS01:0xa4:Entering M.D.
08:55:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2354779254 2686148474 3849180432 2176399459 508864011
08:55:37:WU00:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
08:55:37:WU00:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
08:55:37:WU00:FS00:0x15:Client config unavailable.
08:55:38:WU00:FS00:0x15:Starting GUI Server
08:55:42:WU01:FS01:0xa4:Mapping NT from 8 to 8 
08:55:42:WU01:FS01:0xa4:Completed 0 out of 250000 steps  (0%)
08:56:17:WU01:FS01:0xa4:Completed 2500 out of 250000 steps  (1%)
08:56:40:WU00:FS00:0x15:Setting checkpoint frequency: 250000
08:56:41:WU00:FS00:0x15:Completed         3 out of 25000000 steps (0%).

<SNIP>

09:48:32:WU01:FS01:0xa4:Completed 250000 out of 250000 steps  (100%)
09:48:32:WU01:FS01:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
09:48:33:WU02:FS01:Connecting to assign3.stanford.edu:8080
09:48:35:WU02:FS01:News: Welcome to Folding@Home
09:48:36:WU02:FS01:Assigned to work server 129.74.85.15
09:48:36:WU02:FS01:Requesting new work unit for slot 01: RUNNING smp:8 from 129.74.85.15
09:48:36:WU02:FS01:Connecting to 129.74.85.15:8080
09:48:37:WU02:FS01:Downloading 52.16KiB
09:48:37:WU02:FS01:Download complete
09:48:37:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:OK project:10090 run:28 clone:4 gen:207 core:0xa4 unit:0x000001fc0001329c4dfb52214b1e78c9
09:48:42:WU01:FS01:0xa4:
09:48:42:WU01:FS01:0xa4:Finished Work Unit:
09:48:42:WU01:FS01:0xa4:- Reading up to 596292 from "01/wudata_01.trr": Read 596292
09:48:42:WU01:FS01:0xa4:trr file hash check passed.
09:48:42:WU01:FS01:0xa4:- Reading up to 353100 from "01/wudata_01.xtc": Read 353100
09:48:42:WU01:FS01:0xa4:xtc file hash check passed.
09:48:42:WU01:FS01:0xa4:edr file hash check passed.
09:48:42:WU01:FS01:0xa4:logfile size: 23165
09:48:42:WU01:FS01:0xa4:Leaving Run
09:48:43:WU01:FS01:0xa4:- Writing 978145 bytes of core data to disk...
09:48:43:WU01:FS01:0xa4:Done: 977633 -> 930159 (compressed to 95.1 percent)
09:48:43:WU01:FS01:0xa4:  ... Done.
09:48:44:WU01:FS01:0xa4:- Shutting down core
09:48:44:WU01:FS01:0xa4:
09:48:44:WU01:FS01:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
09:48:44:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
09:48:44:WU01:FS01:Sending unit results: id:01 state:SEND error:OK project:8046 run:211 clone:43 gen:21 core:0xa4 unit:0x0000001c6652edcc4fd940e5c1c72e65
09:48:44:WU01:FS01:Uploading 908.86KiB to 171.67.108.60
09:48:44:WU01:FS01:Connecting to 171.67.108.60:8080
09:48:44:WU02:FS01:Starting
09:48:44:WU02:FS01:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -np 8
09:48:44:WU02:FS01:Started FahCore on PID 4392
09:48:44:Started thread 11 on PID 1740
09:48:44:WU02:FS01:Core PID:4808
09:48:44:WU02:FS01:FahCore 0xa4 started
09:48:44:WU02:FS01:0xa4:
09:48:44:WU02:FS01:0xa4:*------------------------------*
09:48:44:WU02:FS01:0xa4:Folding@Home Gromacs GB Core
09:48:44:WU02:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
09:48:44:WU02:FS01:0xa4:
09:48:44:WU02:FS01:0xa4:Preparing to commence simulation
09:48:44:WU02:FS01:0xa4:- Looking at optimizations...
09:48:44:WU02:FS01:0xa4:- Created dyn
09:48:44:WU02:FS01:0xa4:- Files status OK
09:48:44:WU02:FS01:0xa4:- Expanded 52895 -> 198568 (decompressed 375.4 percent)
09:48:44:WU02:FS01:0xa4:Called DecompressByteArray: compressed_data_size=52895 data_size=198568, decompressed_data_size=198568 diff=0
09:48:44:WU02:FS01:0xa4:- Digital signature verified
09:48:44:WU02:FS01:0xa4:
09:48:44:WU02:FS01:0xa4:Project: 10090 (Run 28, Clone 4, Gen 207)
09:48:44:WU02:FS01:0xa4:
09:48:44:WU02:FS01:0xa4:Assembly optimizations on if available.
09:48:44:WU02:FS01:0xa4:Entering M.D.
09:48:50:WU01:FS01:Upload 42.25%
09:48:50:WU02:FS01:0xa4:Mapping NT from 8 to 8 
09:48:50:WU02:FS01:0xa4:Completed 0 out of 10000000 steps  (0%)
09:48:56:WU01:FS01:Upload 84.50%
09:48:59:WU01:FS01:Upload complete
09:48:59:WU01:FS01:Server responded WORK_ACK (400)
09:48:59:WU01:FS01:Final credit estimate, 699.00 points
09:48:59:WU01:FS01:Cleaning up
09:51:00:WU02:FS01:0xa4:Completed 100000 out of 10000000 steps  (1%)
09:51:35:WU00:FS00:0x15:Completed   3250000 out of 25000000 steps (13%).

<SNIP>

13:22:42:WU00:FS00:0x15:Completed  15750000 out of 25000000 steps (63%).
13:24:33:WU02:FS01:0xa4:Completed 9900000 out of 10000000 steps  (99%)
13:24:33:WU01:FS01:Connecting to assign3.stanford.edu:8080
13:24:34:WU01:FS01:News: Welcome to Folding@Home
13:24:34:WU01:FS01:Assigned to work server 129.74.85.15
13:24:34:WU01:FS01:Requesting new work unit for slot 01: RUNNING smp:8 from 129.74.85.15
13:24:34:WU01:FS01:Connecting to 129.74.85.15:8080
13:24:34:WU01:FS01:Downloading 50.56KiB
13:24:35:WU01:FS01:Download complete
13:24:35:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:OK project:7022 run:3 clone:13 gen:35 core:0xa4 unit:0x000000540001329c4dfba8ce7b0abc43
13:24:35:WU01:FS01:Downloading project 7022 description
13:24:35:WU01:FS01:Connecting to fah-web.stanford.edu:80
13:24:36:WU01:FS01:Project 7022 description downloaded successfully
13:26:43:WU02:FS01:0xa4:Completed 10000000 out of 10000000 steps  (100%)
13:26:43:WU02:FS01:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
13:26:53:WU02:FS01:0xa4:
13:26:53:WU02:FS01:0xa4:Finished Work Unit:
13:26:53:WU02:FS01:0xa4:- Reading up to 2041008 from "02/wudata_01.trr": Read 2041008
13:26:53:WU02:FS01:0xa4:trr file hash check passed.
13:26:53:WU02:FS01:0xa4:- Reading up to 213716 from "02/wudata_01.xtc": Read 213716
13:26:53:WU02:FS01:0xa4:xtc file hash check passed.
13:26:53:WU02:FS01:0xa4:edr file hash check passed.
13:26:53:WU02:FS01:0xa4:logfile size: 79397
13:26:53:WU02:FS01:0xa4:Leaving Run
13:26:55:WU00:FS00:0x15:Completed  16000000 out of 25000000 steps (64%).
13:26:55:WU02:FS01:0xa4:- Writing 2358593 bytes of core data to disk...
13:26:56:WU02:FS01:0xa4:Done: 2358081 -> 1750061 (compressed to 74.2 percent)
13:26:56:WU02:FS01:0xa4:  ... Done.
13:26:56:WU02:FS01:0xa4:- Shutting down core
13:26:56:WU02:FS01:0xa4:
13:26:56:WU02:FS01:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
13:26:56:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
13:26:56:WU02:FS01:Sending unit results: id:02 state:SEND error:OK project:10090 run:28 clone:4 gen:207 core:0xa4 unit:0x000001fc0001329c4dfb52214b1e78c9
13:26:56:WU02:FS01:Uploading 1.67MiB to 129.74.85.15
13:26:56:WU02:FS01:Connecting to 129.74.85.15:8080
13:26:56:WU01:FS01:Starting
13:26:56:WU01:FS01:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -np 8
13:26:56:WU01:FS01:Started FahCore on PID 4952
13:26:56:Started thread 12 on PID 1740
13:26:56:WU01:FS01:Core PID:3024
13:26:56:WU01:FS01:FahCore 0xa4 started
13:26:57:WU01:FS01:0xa4:
13:26:57:WU01:FS01:0xa4:*------------------------------*
13:26:57:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
13:26:57:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
13:26:57:WU01:FS01:0xa4:
13:26:57:WU01:FS01:0xa4:Preparing to commence simulation
13:26:57:WU01:FS01:0xa4:- Looking at optimizations...
13:26:57:WU01:FS01:0xa4:- Created dyn
13:26:57:WU01:FS01:0xa4:- Files status OK
13:26:57:WU01:FS01:0xa4:- Expanded 51261 -> 201536 (decompressed 393.1 percent)
13:26:57:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=51261 data_size=201536, decompressed_data_size=201536 diff=0
13:26:57:WU01:FS01:0xa4:- Digital signature verified
13:26:57:WU01:FS01:0xa4:
13:26:57:WU01:FS01:0xa4:Project: 7022 (Run 3, Clone 13, Gen 35)
13:26:57:WU01:FS01:0xa4:
13:26:57:WU01:FS01:0xa4:Assembly optimizations on if available.
13:26:57:WU01:FS01:0xa4:Entering M.D.
13:27:02:WU02:FS01:Upload 22.46%
13:27:02:WU01:FS01:0xa4:Mapping NT from 8 to 8 
13:27:02:WU01:FS01:0xa4:Completed 0 out of 10000000 steps  (0%)
13:27:08:WU02:FS01:Upload 44.92%
13:27:14:WU02:FS01:Upload 67.39%
13:27:20:WU02:FS01:Upload 93.59%
13:27:23:WU02:FS01:Upload complete
13:27:23:WU02:FS01:Server responded WORK_ACK (400)
13:27:23:WU02:FS01:Final credit estimate, 4421.00 points
13:27:23:WU02:FS01:Cleaning up
13:29:16:WU01:FS01:0xa4:Completed 100000 out of 10000000 steps  (1%)
13:31:09:WU00:FS00:0x15:Completed  16250000 out of 25000000 steps (65%).

<SNIP>

15:53:44:WU01:FS01:0xa4:Completed 6500000 out of 10000000 steps  (65%)
15:54:42:WU00:FS00:0x15:Completed  24750000 out of 25000000 steps (99%).
15:54:43:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
15:54:44:WU02:FS00:News: Welcome to Folding@Home
15:54:44:WU02:FS00:Assigned to work server 171.67.108.143
15:54:44:WU02:FS00:Requesting new work unit for slot 00: RUNNING gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.67.108.143
15:54:44:WU02:FS00:Connecting to 171.67.108.143:8080
15:54:45:WU02:FS00:Downloading 142.81KiB
15:54:47:WU02:FS00:Download complete
15:54:47:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:OK project:8020 run:4 clone:222 gen:80 core:0x15 unit:0x000000696953ee2f4f9679fdc4b0fe66
15:55:59:WU01:FS01:0xa4:Completed 6600000 out of 10000000 steps  (66%)
15:58:13:WU01:FS01:0xa4:Completed 6700000 out of 10000000 steps  (67%)
15:58:56:WU00:FS00:0x15:Completed  25000000 out of 25000000 steps (100%).
15:58:57:WU00:FS00:0x15:Finished fah_main status=0
15:58:57:WU00:FS00:0x15:Successful run
15:58:57:WU00:FS00:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
15:59:07:WU00:FS00:0x15:Reserved 436428 bytes for xtc file; Cosm status=0
15:59:07:WU00:FS00:0x15:Allocated 436428 bytes for xtc file
15:59:07:WU00:FS00:0x15:- Reading up to 436428 from "00/wudata_01.xtc": Read 436428
15:59:07:WU00:FS00:0x15:Read 436428 bytes from xtc file; available packet space=785994036
15:59:07:WU00:FS00:0x15:xtc file hash check passed.
15:59:07:WU00:FS00:0x15:Reserved 54408 54408 785994036 bytes for arc file=<00/wudata_01.trr> Cosm status=0
15:59:07:WU00:FS00:0x15:Allocated 54408 bytes for arc file
15:59:07:WU00:FS00:0x15:- Reading up to 54408 from "00/wudata_01.trr": Read 54408
15:59:07:WU00:FS00:0x15:Read 54408 bytes from arc file; available packet space=785939628
15:59:07:WU00:FS00:0x15:trr file hash check passed.
15:59:07:WU00:FS00:0x15:Allocated 544 bytes for edr file
15:59:07:WU00:FS00:0x15:Read bedfile
15:59:07:WU00:FS00:0x15:edr file hash check passed.
15:59:07:WU00:FS00:0x15:Allocated 32222 bytes for logfile
15:59:07:WU00:FS00:0x15:Read logfile
15:59:07:WU00:FS00:0x15:GuardedRun: success in DynamicWrapper
15:59:07:WU00:FS00:0x15:GuardedRun: done
15:59:07:WU00:FS00:0x15:Run: GuardedRun completed.
15:59:07:WU00:FS00:0x15:+ Opened results file
15:59:07:WU00:FS00:0x15:- Writing 524114 bytes of core data to disk...
15:59:07:WU00:FS00:0x15:Done: 523602 -> 501158 (compressed to 95.7 percent)
15:59:07:WU00:FS00:0x15:  ... Done.
15:59:07:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
15:59:07:WU00:FS00:0x15:Shutting down core 
15:59:07:WU00:FS00:0x15:
15:59:07:WU00:FS00:0x15:Folding@home Core Shutdown: FINISHED_UNIT
15:59:07:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
15:59:07:WU00:FS00:Sending unit results: id:00 state:SEND error:OK project:8020 run:1 clone:145 gen:31 core:0x15 unit:0x0000002c6953ee2f4f96790436da2877
15:59:07:WU00:FS00:Uploading 489.91KiB to 171.67.108.143
15:59:07:WU02:FS00:Starting
15:59:07:WU00:FS00:Connecting to 171.67.108.143:8080
15:59:07:WU02:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
15:59:07:WU02:FS00:Started FahCore on PID 3900
15:59:07:Started thread 13 on PID 1740
15:59:07:WU02:FS00:Core PID:3044
15:59:07:WU02:FS00:FahCore 0x15 started
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:*------------------------------*
15:59:08:WU02:FS00:0x15:Folding@Home GPU Core
15:59:08:WU02:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
15:59:08:WU02:FS00:0x15:Build host             SimbiosNvdWin7
15:59:08:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
15:59:08:WU02:FS00:0x15:Core                   15
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Window's signal control handler registered.
15:59:08:WU02:FS00:0x15:Preparing to commence simulation
15:59:08:WU02:FS00:0x15:- Looking at optimizations...
15:59:08:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
15:59:08:WU02:FS00:0x15:- Created dyn
15:59:08:WU02:FS00:0x15:- Files status OK
15:59:08:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:59:08:WU02:FS00:0x15:- Expanded 145726 -> 660994 (decompressed 453.5 percent)
15:59:08:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=145726 data_size=660994, decompressed_data_size=660994 diff=0
15:59:08:WU02:FS00:0x15:- Digital signature verified
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Project: 8020 (Run 4, Clone 222, Gen 80)
15:59:08:WU02:FS00:0x15:
15:59:08:WU02:FS00:0x15:Assembly optimizations on if available.
15:59:08:WU02:FS00:0x15:Entering M.D.
15:59:10:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1984467589 290667450 1345239047 429439430 2573288279
15:59:10:WU02:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
15:59:10:WU02:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
15:59:10:WU02:FS00:0x15:Client config unavailable.
15:59:10:WU02:FS00:0x15:Starting GUI Server
15:59:13:WU00:FS00:Upload 65.32%
15:59:16:WU00:FS00:Upload complete
15:59:16:WU00:FS00:Server responded WORK_ACK (400)
15:59:16:WU00:FS00:Final credit estimate, 5757.00 points
15:59:16:WU00:FS00:Cleaning up
16:00:11:WU02:FS00:0x15:Setting checkpoint frequency: 250000
16:00:11:WU02:FS00:0x15:Completed         3 out of 25000000 steps (0%).
16:00:29:WU01:FS01:0xa4:Completed 6800000 out of 10000000 steps  (68%)

<SNIP>

17:07:44:WU02:FS00:0x15:Completed   4000000 out of 25000000 steps (16%).
17:09:57:WU01:FS01:0xa4:Completed 9900000 out of 10000000 steps  (99%)
17:09:57:WU00:FS01:Connecting to assign3.stanford.edu:8080
17:09:58:WU00:FS01:News: Welcome to Folding@Home
17:09:58:WU00:FS01:Assigned to work server 129.74.85.15
17:09:58:WU00:FS01:Requesting new work unit for slot 01: RUNNING smp:8 from 129.74.85.15
17:09:58:WU00:FS01:Connecting to 129.74.85.15:8080
17:09:58:WU00:FS01:Downloading 53.29KiB
17:09:59:WU00:FS01:Download complete
17:09:59:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:OK project:7010 run:3 clone:508 gen:23 core:0xa4 unit:0x000000270001329c4dfb962cb2babf93
17:11:58:WU02:FS00:0x15:Completed   4250000 out of 25000000 steps (17%).
17:12:13:WU01:FS01:0xa4:Completed 10000000 out of 10000000 steps  (100%)
17:12:13:WU01:FS01:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
17:12:23:WU01:FS01:0xa4:
17:12:23:WU01:FS01:0xa4:Finished Work Unit:
17:12:23:WU01:FS01:0xa4:- Reading up to 2077368 from "01/wudata_01.trr": Read 2077368
17:12:23:WU01:FS01:0xa4:trr file hash check passed.
17:12:23:WU01:FS01:0xa4:- Reading up to 217112 from "01/wudata_01.xtc": Read 217112
17:12:23:WU01:FS01:0xa4:xtc file hash check passed.
17:12:23:WU01:FS01:0xa4:edr file hash check passed.
17:12:23:WU01:FS01:0xa4:logfile size: 79285
17:12:23:WU01:FS01:0xa4:Leaving Run
17:12:27:WU01:FS01:0xa4:- Writing 2398237 bytes of core data to disk...
17:12:27:WU01:FS01:0xa4:Done: 2397725 -> 1638404 (compressed to 68.3 percent)
17:12:27:WU01:FS01:0xa4:  ... Done.
17:12:28:WU01:FS01:0xa4:- Shutting down core
17:12:28:WU01:FS01:0xa4:
17:12:28:WU01:FS01:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
17:12:28:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:12:28:WU01:FS01:Sending unit results: id:01 state:SEND error:OK project:7022 run:3 clone:13 gen:35 core:0xa4 unit:0x000000540001329c4dfba8ce7b0abc43
17:12:28:WU01:FS01:Uploading 1.56MiB to 129.74.85.15
17:12:28:WU01:FS01:Connecting to 129.74.85.15:8080
17:12:28:WU00:FS01:Starting
17:12:28:WU00:FS01:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -np 8
17:12:28:WU00:FS01:Started FahCore on PID 4148
17:12:28:Started thread 14 on PID 1740
17:12:28:WU00:FS01:Core PID:3716
17:12:28:WU00:FS01:FahCore 0xa4 started
17:12:28:WU00:FS01:0xa4:
17:12:28:WU00:FS01:0xa4:*------------------------------*
17:12:28:WU00:FS01:0xa4:Folding@Home Gromacs GB Core
17:12:28:WU00:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
17:12:28:WU00:FS01:0xa4:
17:12:28:WU00:FS01:0xa4:Preparing to commence simulation
17:12:28:WU00:FS01:0xa4:- Looking at optimizations...
17:12:28:WU00:FS01:0xa4:- Created dyn
17:12:28:WU00:FS01:0xa4:- Files status OK
17:12:28:WU00:FS01:0xa4:- Expanded 54061 -> 208564 (decompressed 385.7 percent)
17:12:28:WU00:FS01:0xa4:Called DecompressByteArray: compressed_data_size=54061 data_size=208564, decompressed_data_size=208564 diff=0
17:12:28:WU00:FS01:0xa4:- Digital signature verified
17:12:28:WU00:FS01:0xa4:
17:12:28:WU00:FS01:0xa4:Project: 7010 (Run 3, Clone 508, Gen 23)
17:12:28:WU00:FS01:0xa4:
17:12:28:WU00:FS01:0xa4:Assembly optimizations on if available.
17:12:28:WU00:FS01:0xa4:Entering M.D.
17:12:34:WU01:FS01:Upload 23.99%
17:12:34:WU00:FS01:0xa4:Mapping NT from 8 to 8 
17:12:34:WU00:FS01:0xa4:Completed 0 out of 10000000 steps  (0%)
17:12:40:WU01:FS01:Upload 47.98%
17:12:46:WU01:FS01:Upload 75.98%
17:12:52:WU01:FS01:Upload 99.97%
17:12:53:WU01:FS01:Upload complete
17:12:53:WU01:FS01:Server responded WORK_ACK (400)
17:12:53:WU01:FS01:Final credit estimate, 4328.00 points
17:12:53:WU01:FS01:Cleaning up
17:14:55:WU00:FS01:0xa4:Completed 100000 out of 10000000 steps  (1%)
17:16:11:WU02:FS00:0x15:Completed   4500000 out of 25000000 steps (18%).

<SNIP>

18:15:18:WU02:FS00:0x15:Completed   8000000 out of 25000000 steps (32%).
18:15:36:WU00:FS01:0xa4:Completed 2700000 out of 10000000 steps  (27%)
18:16:05:FS00:Finishing
18:16:05:FS01:Finishing
18:17:52:WU00:FS01:0xa4:Completed 2800000 out of 10000000 steps  (28%)
18:19:32:WU02:FS00:0x15:Completed   8250000 out of 25000000 steps (33%).
18:20:07:WU00:FS01:0xa4:Completed 2900000 out of 10000000 steps  (29%)
18:22:21:WU00:FS01:0xa4:Completed 3000000 out of 10000000 steps  (30%)
18:23:45:WU02:FS00:0x15:Completed   8500000 out of 25000000 steps (34%).
18:24:37:WU00:FS01:0xa4:Completed 3100000 out of 10000000 steps  (31%)
18:26:51:WU00:FS01:0xa4:Completed 3200000 out of 10000000 steps  (32%)
18:27:59:WU02:FS00:0x15:Completed   8750000 out of 25000000 steps (35%).
18:29:06:WU00:FS01:0xa4:Completed 3300000 out of 10000000 steps  (33%)
18:31:20:WU00:FS01:0xa4:Completed 3400000 out of 10000000 steps  (34%)
18:32:13:WU02:FS00:0x15:Completed   9000000 out of 25000000 steps (36%).
18:33:35:WU00:FS01:0xa4:Completed 3500000 out of 10000000 steps  (35%)
18:35:50:WU00:FS01:0xa4:Completed 3600000 out of 10000000 steps  (36%)
18:36:26:WU02:FS00:0x15:Completed   9250000 out of 25000000 steps (37%).
18:38:05:WU00:FS01:0xa4:Completed 3700000 out of 10000000 steps  (37%)
18:40:19:WU00:FS01:0xa4:Completed 3800000 out of 10000000 steps  (38%)
18:40:40:WU02:FS00:0x15:Completed   9500000 out of 25000000 steps (38%).
18:42:34:WU00:FS01:0xa4:Completed 3900000 out of 10000000 steps  (39%)
18:44:48:WU00:FS01:0xa4:Completed 4000000 out of 10000000 steps  (40%)
18:44:54:WU02:FS00:0x15:Completed   9750000 out of 25000000 steps (39%).
18:47:03:WU00:FS01:0xa4:Completed 4100000 out of 10000000 steps  (41%)
18:49:07:WU02:FS00:0x15:Completed  10000000 out of 25000000 steps (40%).
18:49:18:WU00:FS01:0xa4:Completed 4200000 out of 10000000 steps  (42%)
18:51:32:WU00:FS01:0xa4:Completed 4300000 out of 10000000 steps  (43%)
18:53:21:WU02:FS00:0x15:Completed  10250000 out of 25000000 steps (41%).
18:53:47:WU00:FS01:0xa4:Completed 4400000 out of 10000000 steps  (44%)
18:56:02:WU00:FS01:0xa4:Completed 4500000 out of 10000000 steps  (45%)
18:57:35:WU02:FS00:0x15:Completed  10500000 out of 25000000 steps (42%).
18:58:17:WU00:FS01:0xa4:Completed 4600000 out of 10000000 steps  (46%)
19:00:32:WU00:FS01:0xa4:Completed 4700000 out of 10000000 steps  (47%)
19:01:48:WU02:FS00:0x15:Completed  10750000 out of 25000000 steps (43%).
19:02:46:WU00:FS01:0xa4:Completed 4800000 out of 10000000 steps  (48%)
19:05:01:WU00:FS01:0xa4:Completed 4900000 out of 10000000 steps  (49%)
19:06:02:WU02:FS00:0x15:Completed  11000000 out of 25000000 steps (44%).
19:07:15:WU00:FS01:0xa4:Completed 5000000 out of 10000000 steps  (50%)
19:09:30:WU00:FS01:0xa4:Completed 5100000 out of 10000000 steps  (51%)
19:10:16:WU02:FS00:0x15:Completed  11250000 out of 25000000 steps (45%).
19:11:45:WU00:FS01:0xa4:Completed 5200000 out of 10000000 steps  (52%)
19:13:59:WU00:FS01:0xa4:Completed 5300000 out of 10000000 steps  (53%)
19:14:30:WU02:FS00:0x15:Completed  11500000 out of 25000000 steps (46%).
19:16:14:WU00:FS01:0xa4:Completed 5400000 out of 10000000 steps  (54%)
19:18:28:WU00:FS01:0xa4:Completed 5500000 out of 10000000 steps  (55%)
19:18:43:WU02:FS00:0x15:Completed  11750000 out of 25000000 steps (47%).
19:20:43:WU00:FS01:0xa4:Completed 5600000 out of 10000000 steps  (56%)
19:22:57:WU00:FS01:0xa4:Completed 5700000 out of 10000000 steps  (57%)
19:22:57:WU02:FS00:0x15:Completed  12000000 out of 25000000 steps (48%).
19:25:12:WU00:FS01:0xa4:Completed 5800000 out of 10000000 steps  (58%)
19:27:10:WU02:FS00:0x15:Completed  12250000 out of 25000000 steps (49%).
19:27:26:WU00:FS01:0xa4:Completed 5900000 out of 10000000 steps  (59%)
19:29:40:WU00:FS01:0xa4:Completed 6000000 out of 10000000 steps  (60%)
19:31:24:WU02:FS00:0x15:Completed  12500000 out of 25000000 steps (50%).
19:31:55:WU00:FS01:0xa4:Completed 6100000 out of 10000000 steps  (61%)
19:34:10:WU00:FS01:0xa4:Completed 6200000 out of 10000000 steps  (62%)
19:35:38:WU02:FS00:0x15:Completed  12750000 out of 25000000 steps (51%).
19:36:25:WU00:FS01:0xa4:Completed 6300000 out of 10000000 steps  (63%)
19:38:39:WU00:FS01:0xa4:Completed 6400000 out of 10000000 steps  (64%)
19:39:51:WU02:FS00:0x15:Completed  13000000 out of 25000000 steps (52%).
19:40:54:WU00:FS01:0xa4:Completed 6500000 out of 10000000 steps  (65%)
19:43:08:WU00:FS01:0xa4:Completed 6600000 out of 10000000 steps  (66%)
19:44:05:WU02:FS00:0x15:Completed  13250000 out of 25000000 steps (53%).
19:45:23:WU00:FS01:0xa4:Completed 6700000 out of 10000000 steps  (67%)
19:47:37:WU00:FS01:0xa4:Completed 6800000 out of 10000000 steps  (68%)
19:48:19:WU02:FS00:0x15:Completed  13500000 out of 25000000 steps (54%).
19:49:52:WU00:FS01:0xa4:Completed 6900000 out of 10000000 steps  (69%)
19:52:07:WU00:FS01:0xa4:Completed 7000000 out of 10000000 steps  (70%)
19:52:32:WU02:FS00:0x15:Completed  13750000 out of 25000000 steps (55%).
19:54:22:WU00:FS01:0xa4:Completed 7100000 out of 10000000 steps  (71%)
19:56:37:WU00:FS01:0xa4:Completed 7200000 out of 10000000 steps  (72%)
19:56:46:WU02:FS00:0x15:Completed  14000000 out of 25000000 steps (56%).
19:58:51:WU00:FS01:0xa4:Completed 7300000 out of 10000000 steps  (73%)
20:01:00:WU02:FS00:0x15:Completed  14250000 out of 25000000 steps (57%).
20:01:07:WU00:FS01:0xa4:Completed 7400000 out of 10000000 steps  (74%)
20:03:21:WU00:FS01:0xa4:Completed 7500000 out of 10000000 steps  (75%)
20:05:13:WU02:FS00:0x15:Completed  14500000 out of 25000000 steps (58%).
20:05:36:WU00:FS01:0xa4:Completed 7600000 out of 10000000 steps  (76%)
20:07:50:WU00:FS01:0xa4:Completed 7700000 out of 10000000 steps  (77%)
20:09:27:WU02:FS00:0x15:Completed  14750000 out of 25000000 steps (59%).
20:09:27:WU02:FS00:0x15:mdrun_gpu returned 52
20:09:27:WU02:FS00:0x15:NANs detected on GPU
20:09:27:WU02:FS00:0x15:
20:09:27:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
20:09:28:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
20:09:28:WU02:FS00:Starting
20:09:28:WU02:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
20:09:28:WU02:FS00:Started FahCore on PID 4192
20:09:28:Started thread 15 on PID 1740
20:09:28:WU02:FS00:Core PID:1100
20:09:28:WU02:FS00:FahCore 0x15 started
20:09:28:WU02:FS00:0x15:
20:09:28:WU02:FS00:0x15:*------------------------------*
20:09:28:WU02:FS00:0x15:Folding@Home GPU Core
20:09:28:WU02:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
20:09:28:WU02:FS00:0x15:Build host             SimbiosNvdWin7
20:09:28:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
20:09:28:WU02:FS00:0x15:Core                   15
20:09:28:WU02:FS00:0x15:
20:09:28:WU02:FS00:0x15:Window's signal control handler registered.
20:09:28:WU02:FS00:0x15:Preparing to commence simulation
20:09:28:WU02:FS00:0x15:- Looking at optimizations...
20:09:28:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
20:09:28:WU02:FS00:0x15:- Created dyn
20:09:28:WU02:FS00:0x15:- Files status OK
20:09:28:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
20:09:28:WU02:FS00:0x15:- Expanded 145726 -> 660994 (decompressed 453.5 percent)
20:09:28:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=145726 data_size=660994, decompressed_data_size=660994 diff=0
20:09:29:WU02:FS00:0x15:- Digital signature verified
20:09:29:WU02:FS00:0x15:
20:09:29:WU02:FS00:0x15:Project: 8020 (Run 4, Clone 222, Gen 80)
20:09:29:WU02:FS00:0x15:
20:09:29:WU02:FS00:0x15:Assembly optimizations on if available.
20:09:29:WU02:FS00:0x15:Entering M.D.
20:09:31:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1984467589 290667450 1345239047 429439430 2573288279
20:09:31:WU02:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
20:09:31:WU02:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
20:09:31:WU02:FS00:0x15:Client config unavailable.
20:09:31:WU02:FS00:0x15:Starting GUI Server
20:10:04:WU00:FS01:0xa4:Completed 7800000 out of 10000000 steps  (78%)
20:10:33:WARNING:WU02:FS00:Detected clock skew, adjusting time estimates
20:10:33:WU02:FS00:0x15:Setting checkpoint frequency: 250000
20:10:33:WU02:FS00:0x15:Completed         3 out of 25000000 steps (0%).
20:12:19:WU00:FS01:0xa4:Completed 7900000 out of 10000000 steps  (79%)
20:14:34:WU00:FS01:0xa4:Completed 8000000 out of 10000000 steps  (80%)
20:14:45:WU02:FS00:0x15:Completed    250000 out of 25000000 steps (1%).
20:16:49:WU00:FS01:0xa4:Completed 8100000 out of 10000000 steps  (81%)
20:18:59:WU02:FS00:0x15:Completed    500000 out of 25000000 steps (2%).
20:19:04:WU00:FS01:0xa4:Completed 8200000 out of 10000000 steps  (82%)
20:21:21:WU00:FS01:0xa4:Completed 8300000 out of 10000000 steps  (83%)
20:23:13:WU02:FS00:0x15:Completed    750000 out of 25000000 steps (3%).
20:23:41:WU00:FS01:0xa4:Completed 8400000 out of 10000000 steps  (84%)
20:26:00:WU00:FS01:0xa4:Completed 8500000 out of 10000000 steps  (85%)
20:27:26:WU02:FS00:0x15:Completed   1000000 out of 25000000 steps (4%).
20:28:20:WU00:FS01:0xa4:Completed 8600000 out of 10000000 steps  (86%)
20:30:39:WU00:FS01:0xa4:Completed 8700000 out of 10000000 steps  (87%)
20:31:39:WU02:FS00:0x15:Completed   1250000 out of 25000000 steps (5%).
20:32:58:WU00:FS01:0xa4:Completed 8800000 out of 10000000 steps  (88%)
20:35:17:WU00:FS01:0xa4:Completed 8900000 out of 10000000 steps  (89%)
20:35:53:WU02:FS00:0x15:Completed   1500000 out of 25000000 steps (6%).
20:37:36:WU00:FS01:0xa4:Completed 9000000 out of 10000000 steps  (90%)
20:39:56:WU00:FS01:0xa4:Completed 9100000 out of 10000000 steps  (91%)
20:40:06:WU02:FS00:0x15:Completed   1750000 out of 25000000 steps (7%).
20:42:12:WU00:FS01:0xa4:Completed 9200000 out of 10000000 steps  (92%)
20:44:20:WU02:FS00:0x15:Completed   2000000 out of 25000000 steps (8%).
20:44:27:WU00:FS01:0xa4:Completed 9300000 out of 10000000 steps  (93%)
20:46:41:WU00:FS01:0xa4:Completed 9400000 out of 10000000 steps  (94%)
20:48:33:WU02:FS00:0x15:Completed   2250000 out of 25000000 steps (9%).
20:48:56:WU00:FS01:0xa4:Completed 9500000 out of 10000000 steps  (95%)
20:51:10:WU00:FS01:0xa4:Completed 9600000 out of 10000000 steps  (96%)
20:52:47:WU02:FS00:0x15:Completed   2500000 out of 25000000 steps (10%).
20:53:26:WU00:FS01:0xa4:Completed 9700000 out of 10000000 steps  (97%)
******************************** Date: 23/06/12 ********************************
20:55:41:WU00:FS01:0xa4:Completed 9800000 out of 10000000 steps  (98%)
20:57:01:WU02:FS00:0x15:Completed   2750000 out of 25000000 steps (11%).
20:57:56:WU00:FS01:0xa4:Completed 9900000 out of 10000000 steps  (99%)
21:00:10:WU00:FS01:0xa4:Completed 10000000 out of 10000000 steps  (100%)
21:00:10:WU00:FS01:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
21:00:21:WU00:FS01:0xa4:
21:00:21:WU00:FS01:0xa4:Finished Work Unit:
21:00:21:WU00:FS01:0xa4:- Reading up to 2153724 from "00/wudata_01.trr": Read 2153724
21:00:21:WU00:FS01:0xa4:trr file hash check passed.
21:00:21:WU00:FS01:0xa4:- Reading up to 223732 from "00/wudata_01.xtc": Read 223732
21:00:21:WU00:FS01:0xa4:xtc file hash check passed.
21:00:21:WU00:FS01:0xa4:edr file hash check passed.
21:00:21:WU00:FS01:0xa4:logfile size: 79349
21:00:21:WU00:FS01:0xa4:Leaving Run
21:00:24:WU00:FS01:0xa4:- Writing 2481277 bytes of core data to disk...
21:00:24:WU00:FS01:0xa4:Done: 2480765 -> 1822625 (compressed to 73.4 percent)
21:00:24:WU00:FS01:0xa4:  ... Done.
21:00:24:WU00:FS01:0xa4:- Shutting down core
21:00:24:WU00:FS01:0xa4:
21:00:24:WU00:FS01:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
21:00:25:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:00:25:WU00:FS01:Sending unit results: id:00 state:SEND error:OK project:7010 run:3 clone:508 gen:23 core:0xa4 unit:0x000000270001329c4dfb962cb2babf93
21:00:25:WU00:FS01:Uploading 1.74MiB to 129.74.85.15
21:00:25:WU00:FS01:Connecting to 129.74.85.15:8080
21:00:31:WU00:FS01:Upload 21.57%
21:00:37:WU00:FS01:Upload 46.73%
21:00:43:WU00:FS01:Upload 68.30%
21:00:49:WU00:FS01:Upload 89.87%
21:00:52:WU00:FS01:Upload complete
21:00:52:WU00:FS01:Server responded WORK_ACK (400)
21:00:52:WU00:FS01:Final credit estimate, 4303.00 points
21:00:52:WU00:FS01:Cleaning up
21:01:15:WU02:FS00:0x15:Completed   3000000 out of 25000000 steps (12%).

<SNIP>

23:09:14:WU02:FS00:0x15:Completed  10500000 out of 25000000 steps (42%).
23:09:14:WU02:FS00:0x15:mdrun_gpu returned 52
23:09:14:WU02:FS00:0x15:NANs detected on GPU
23:09:14:WU02:FS00:0x15:
23:09:15:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
23:09:15:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
23:09:15:WU02:FS00:Starting
23:09:15:WU02:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 701 -lifeline 1740 -checkpoint 15 -gpu 0
23:09:15:WU02:FS00:Started FahCore on PID 4536
23:09:15:Started thread 16 on PID 1740
23:09:15:WU02:FS00:Core PID:3776
23:09:15:WU02:FS00:FahCore 0x15 started
23:09:16:WU02:FS00:0x15:
23:09:16:WU02:FS00:0x15:*------------------------------*
23:09:16:WU02:FS00:0x15:Folding@Home GPU Core
23:09:16:WU02:FS00:0x15:Version                2.22 (Thu Dec 8 17:08:05 PST 2011)
23:09:16:WU02:FS00:0x15:Build host             SimbiosNvdWin7
23:09:16:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
23:09:16:WU02:FS00:0x15:Core                   15
23:09:16:WU02:FS00:0x15:
23:09:16:WU02:FS00:0x15:Window's signal control handler registered.
23:09:16:WU02:FS00:0x15:Preparing to commence simulation
23:09:16:WU02:FS00:0x15:- Looking at optimizations...
23:09:16:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
23:09:16:WU02:FS00:0x15:- Created dyn
23:09:16:WU02:FS00:0x15:- Files status OK
23:09:16:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
23:09:16:WU02:FS00:0x15:- Expanded 145726 -> 660994 (decompressed 453.5 percent)
23:09:16:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=145726 data_size=660994, decompressed_data_size=660994 diff=0
23:09:16:WU02:FS00:0x15:- Digital signature verified
23:09:16:WU02:FS00:0x15:
23:09:16:WU02:FS00:0x15:Project: 8020 (Run 4, Clone 222, Gen 80)
23:09:16:WU02:FS00:0x15:
23:09:16:WU02:FS00:0x15:Assembly optimizations on if available.
23:09:16:WU02:FS00:0x15:Entering M.D.
23:09:17:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1984467589 290667450 1345239047 429439430 2573288279
23:09:17:WU02:FS00:0x15:GPU device info: vendor=0 device=0 name=<NA> match=0
23:09:17:WU02:FS00:0x15:Working on Gromacs Runs On Most of All Computer Systems
23:09:17:WU02:FS00:0x15:Client config unavailable.
23:09:18:WU02:FS00:0x15:Starting GUI Server
23:10:19:WARNING:WU02:FS00:Detected clock skew, adjusting time estimates
23:10:19:WU02:FS00:0x15:Setting checkpoint frequency: 250000
23:10:19:WU02:FS00:0x15:Completed         3 out of 25000000 steps (0%).

<SNIP>

00:39:54:WU02:FS00:0x15:Completed   5250000 out of 25000000 steps (21%).
00:44:10:WU02:FS00:0x15:Completed   5500000 out of 25000000 steps (22%).
00:45:03:Lost lifeline PID 3312, exiting
00:45:04:FS00:Shutting core down
00:45:04:Server connection id=1 ended
00:45:06:Clean exit
00:45:06:WU02:FS00:0x15:Client no longer detected. Shutting down core 
00:45:06:WU02:FS00:0x15:
00:45:06:WU02:FS00:0x15:Folding@home Core Shutdown: CLIENT_DIED

I have now removed my GPU again (see also the HFM thread starting on page 26 (viewtopic.php?f=14&t=9903).

[7im]

NaNs are almost always hardware related. Failing or overclocked too much, or too hot, or not enough voltage. And that fact the same WU failed in a different place nails it for sure. Bad WUs almost always fail in the same place.

[Fahrenheit451]

The card is a brand new replacement for my former GTX 560 Ti (same factory OCed model from Gigabyte). I added the card yesterday to the PC. When FAH is running the card is @ 73°C which should not be too hot. Voltage should be ok: The card is connected with two 6-pin connectors to the 580 W PSU (bequiet Straight Power E8 with cable management) on its own rail and I didn't change anything in the BIOS. Let's see how the card will work within the next few days. I hope I didn't get a faulty piece of hardware. It's the second replacement of my original GTX 560 Ti within 11 months.

Thank you for the information that bad WUs mostly fail in the same place. That was new to me.

[7im]

To be 100% percent sure (instead of 98% sure) of a bad WU vs. a hardware issue is to wait for a Mod to recheck that WU in a few days and see if anyone else can complete it successfully.

[bruce]

No need to wait.

Someone else had WU (P8020 R4 C222 G80) added to the stats database on 2012-06-24 11:08:35 for 5757 points of credit.

If they can complete it successfully and you can't, something is wrong with your system.