Page 1 of 1

5765 NANs despite data del, reinstall, latest NV drv

Posted: Thu Feb 19, 2009 10:47 am
by gue22
Intel Q9300, 35 series chipset, 8GB RAM
9800GTX, 512MB
Vista Ult fully patched, machine left alone but BOINC projects on CPU

Been running beautifully since June. All of a sudden NANs for two weeks now. Uninstall, data removal, latest client reinstall, NV latest 182.06 drv no cure.
Moved to BOINC GPU-grid for the time beeing, because this is just too frustrating.
Greetings
G.

Code: Select all

--- Opening Log file [February 18 19:53:09 UTC] 


# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\gy\AppData\Roaming\Folding@home-gpu


[19:53:09] - Ask before connecting: No
[19:53:09] - User name: gue22 (Team 0)
[19:53:09] - User ID: 364F170142FAF600
[19:53:09] - Machine ID: 2
[19:53:09] 
[19:53:09] Loaded queue successfully.
[19:53:09] Initialization complete
[19:53:09] - Preparing to get new work unit...
[19:53:09] + Attempting to get work packet
[19:53:09] - Connecting to assignment server
[19:53:10] - Successful: assigned to (171.67.108.11).
[19:53:10] + News From Folding@Home: GPU folding beta
[19:53:10] Loaded queue successfully.
[19:53:18] + Closed connections
[19:53:18] 
[19:53:18] + Processing work unit
[19:53:18] Core required: FahCore_11.exe
[19:53:18] Core found.
[19:53:18] Working on queue slot 04 [February 18 19:53:18 UTC]
[19:53:18] + Working ...
[19:53:18] 
[19:53:18] *------------------------------*
[19:53:18] Folding@Home GPU Core - Beta
[19:53:18] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[19:53:18] 
[19:53:18] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[19:53:18] Build host: amoeba
[19:53:18] Board Type: Nvidia
[19:53:18] Core      : 
[19:53:18] Preparing to commence simulation
[19:53:18] - Looking at optimizations...
[19:53:19] - Created dyn
[19:53:19] - Files status OK
[19:53:19] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[19:53:19] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[19:53:19] - Digital signature verified
[19:53:19] 
[19:53:19] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:53:19] 
[19:53:20] Assembly optimizations on if available.
[19:53:20] Entering M.D.
[19:53:31] Working on Protein
[19:53:37] Client config found, loading data.
[19:53:37] Starting GUI Server
[19:53:38] mdrun_gpu returned 
[19:53:38] NANs detected on GPU
[19:53:38] 
[19:53:38] Folding@home Core Shutdown: UNSTABLE_MACHINE
[19:53:42] CoreStatus = 7A (122)
[19:53:42] Sending work to server
[19:53:42] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:53:42] - Error: Could not get length of results file work/wuresults_04.dat
[19:53:42] - Error: Could not read unit 04 file. Removing from queue.
[19:53:42] - Preparing to get new work unit...
[19:53:42] + Attempting to get work packet
[19:53:42] - Connecting to assignment server
[19:53:43] - Successful: assigned to (171.67.108.11).
[19:53:43] + News From Folding@Home: GPU folding beta
[19:53:43] Loaded queue successfully.
[19:53:45] + Closed connections
[19:53:50] 
[19:53:50] + Processing work unit
[19:53:50] Core required: FahCore_11.exe
[19:53:50] Core found.
[19:53:50] Working on queue slot 05 [February 18 19:53:50 UTC]
[19:53:50] + Working ...
[19:53:50] 
[19:53:50] *------------------------------*
[19:53:50] Folding@Home GPU Core - Beta
[19:53:50] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[19:53:50] 
[19:53:50] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[19:53:50] Build host: amoeba
[19:53:50] Board Type: Nvidia
[19:53:50] Core      : 
[19:53:50] Preparing to commence simulation
[19:53:50] - Looking at optimizations...
[19:53:50] - Created dyn
[19:53:50] - Files status OK
[19:53:50] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[19:53:50] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[19:53:50] - Digital signature verified
[19:53:50] 
[19:53:50] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:53:50] 
[19:53:50] Assembly optimizations on if available.
[19:53:50] Entering M.D.
[19:53:57] Working on Protein
[19:54:02] Client config found, loading data.
[19:54:02] Starting GUI Server
[19:54:02] mdrun_gpu returned 
[19:54:02] NANs detected on GPU
[19:54:02] 
[19:54:02] Folding@home Core Shutdown: UNSTABLE_MACHINE
[19:54:06] CoreStatus = 7A (122)
[19:54:06] Sending work to server
[19:54:06] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:54:06] - Error: Could not get length of results file work/wuresults_05.dat
[19:54:06] - Error: Could not read unit 05 file. Removing from queue.
[19:54:06] - Preparing to get new work unit...
[19:54:06] + Attempting to get work packet
[19:54:06] - Connecting to assignment server
[19:54:07] - Successful: assigned to (171.67.108.11).
[19:54:07] + News From Folding@Home: GPU folding beta
[19:54:07] Loaded queue successfully.
[19:54:09] + Closed connections
[19:54:14] 
[19:54:14] + Processing work unit
[19:54:14] Core required: FahCore_11.exe
[19:54:14] Core found.
[19:54:14] Working on queue slot 06 [February 18 19:54:14 UTC]
[19:54:14] + Working ...
[19:54:14] 
[19:54:14] *------------------------------*
[19:54:14] Folding@Home GPU Core - Beta
[19:54:14] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[19:54:14] 
[19:54:14] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[19:54:14] Build host: amoeba
[19:54:14] Board Type: Nvidia
[19:54:14] Core      : 
[19:54:14] Preparing to commence simulation
[19:54:14] - Looking at optimizations...
[19:54:14] - Created dyn
[19:54:14] - Files status OK
[19:54:14] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[19:54:14] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[19:54:14] - Digital signature verified
[19:54:14] 
[19:54:14] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:54:14] 
[19:54:14] Assembly optimizations on if available.
[19:54:14] Entering M.D.
[19:54:21] Working on Protein
[19:54:28] Client config found, loading data.
[19:54:28] Starting GUI Server
[19:54:29] mdrun_gpu returned 
[19:54:29] NANs detected on GPU
[19:54:29] 
[19:54:29] Folding@home Core Shutdown: UNSTABLE_MACHINE
[19:54:32] CoreStatus = 7A (122)
[19:54:32] Sending work to server
[19:54:32] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:54:32] - Error: Could not get length of results file work/wuresults_06.dat
[19:54:32] - Error: Could not read unit 06 file. Removing from queue.
[19:54:32] - Preparing to get new work unit...
[19:54:32] + Attempting to get work packet
[19:54:32] - Connecting to assignment server
[19:54:33] - Successful: assigned to (171.67.108.11).
[19:54:33] + News From Folding@Home: GPU folding beta
[19:54:33] Loaded queue successfully.
[19:54:35] + Closed connections
[19:54:40] 
[19:54:40] + Processing work unit
[19:54:40] Core required: FahCore_11.exe
[19:54:40] Core found.
[19:54:40] Working on queue slot 07 [February 18 19:54:40 UTC]
[19:54:40] + Working ...
[19:54:40] 
[19:54:40] *------------------------------*
[19:54:40] Folding@Home GPU Core - Beta
[19:54:40] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[19:54:40] 
[19:54:40] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[19:54:40] Build host: amoeba
[19:54:40] Board Type: Nvidia
[19:54:40] Core      : 
[19:54:40] Preparing to commence simulation
[19:54:40] - Looking at optimizations...
[19:54:40] - Created dyn
[19:54:40] - Files status OK
[19:54:40] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[19:54:40] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[19:54:40] - Digital signature verified
[19:54:40] 
[19:54:40] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:54:40] 
[19:54:40] Assembly optimizations on if available.
[19:54:40] Entering M.D.
[19:54:47] Working on Protein
[19:54:53] Client config found, loading data.
[19:54:53] Starting GUI Server
[19:54:53] mdrun_gpu returned 
[19:54:53] NANs detected on GPU
[19:54:53] 
[19:54:53] Folding@home Core Shutdown: UNSTABLE_MACHINE
[19:54:56] CoreStatus = 7A (122)
[19:54:56] Sending work to server
[19:54:56] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:54:56] - Error: Could not get length of results file work/wuresults_07.dat
[19:54:56] - Error: Could not read unit 07 file. Removing from queue.
[19:54:56] - Preparing to get new work unit...
[19:54:56] + Attempting to get work packet
[19:54:56] - Connecting to assignment server
[19:54:57] - Successful: assigned to (171.67.108.11).
[19:54:57] + News From Folding@Home: GPU folding beta
[19:54:57] Loaded queue successfully.
[19:54:59] + Closed connections
[19:55:04] 
[19:55:04] + Processing work unit
[19:55:04] Core required: FahCore_11.exe
[19:55:04] Core found.
[19:55:04] Working on queue slot 08 [February 18 19:55:04 UTC]
[19:55:04] + Working ...
[19:55:04] 
[19:55:04] *------------------------------*
[19:55:04] Folding@Home GPU Core - Beta
[19:55:04] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[19:55:04] 
[19:55:04] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[19:55:04] Build host: amoeba
[19:55:04] Board Type: Nvidia
[19:55:04] Core      : 
[19:55:04] Preparing to commence simulation
[19:55:04] - Looking at optimizations...
[19:55:04] - Created dyn
[19:55:04] - Files status OK
[19:55:04] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[19:55:04] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[19:55:04] - Digital signature verified
[19:55:04] 
[19:55:04] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:55:04] 
[19:55:04] Assembly optimizations on if available.
[19:55:04] Entering M.D.
[19:55:11] Working on Protein
[19:55:16] Client config found, loading data.
[19:55:16] Starting GUI Server
[19:55:16] mdrun_gpu returned 
[19:55:16] NANs detected on GPU
[19:55:16] 
[19:55:16] Folding@home Core Shutdown: UNSTABLE_MACHINE
[19:55:20] CoreStatus = 7A (122)
[19:55:20] Sending work to server
[19:55:20] Project: 5765 (Run 7, Clone 120, Gen 188)
[19:55:20] - Error: Could not get length of results file work/wuresults_08.dat
[19:55:20] - Error: Could not read unit 08 file. Removing from queue.
[19:55:20] EUE limit exceeded. Pausing 24 hours.


--- Opening Log file [February 18 20:41:12 UTC] 


# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\gy\AppData\Roaming\Folding@home-gpu


[20:41:12] - Ask before connecting: No
[20:41:12] - User name: gue22 (Team 0)
[20:41:12] - User ID: 364F170142FAF600
[20:41:12] - Machine ID: 2
[20:41:12] 
[20:41:12] Loaded queue successfully.
[20:41:13] Initialization complete
[20:41:13] - Preparing to get new work unit...
[20:41:13] + Attempting to get work packet
[20:41:13] - Connecting to assignment server
[20:41:13] - Successful: assigned to (171.67.108.11).
[20:41:13] + News From Folding@Home: GPU folding beta
[20:41:13] Loaded queue successfully.
[20:41:15] + Closed connections
[20:41:15] 
[20:41:15] + Processing work unit
[20:41:15] Core required: FahCore_11.exe
[20:41:15] Core found.
[20:41:15] Working on queue slot 09 [February 18 20:41:15 UTC]
[20:41:15] + Working ...
[20:41:15] 
[20:41:15] *------------------------------*
[20:41:15] Folding@Home GPU Core - Beta
[20:41:15] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[20:41:15] 
[20:41:15] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:41:15] Build host: amoeba
[20:41:15] Board Type: Nvidia
[20:41:15] Core      : 
[20:41:15] Preparing to commence simulation
[20:41:15] - Looking at optimizations...
[20:41:16] - Created dyn
[20:41:16] - Files status OK
[20:41:16] - Expanded 46739 -> 252912 (decompressed 541.1 percent)
[20:41:16] Called DecompressByteArray: compressed_data_size=46739 data_size=252912, decompressed_data_size=252912 diff=0
[20:41:16] - Digital signature verified
[20:41:16] 
[20:41:16] Project: 5765 (Run 7, Clone 120, Gen 188)
[20:41:16] 
[20:41:16] Assembly optimizations on if available.
[20:41:16] Entering M.D.
[20:41:22] Working on Protein
[20:41:24] Client config found, loading data.
[20:41:24] Starting GUI Server
[20:41:24] mdrun_gpu returned 
[20:41:24] NANs detected on GPU
[20:41:24] 
[20:41:24] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:41:29] CoreStatus = 7A (122)
[20:41:29] Sending work to server
[20:41:29] Project: 5765 (Run 7, Clone 120, Gen 188)
[20:41:29] - Error: Could not get length of results file work/wuresults_09.dat
[20:41:29] - Error: Could not read unit 09 file. Removing from queue.
[20:41:29] - Preparing to get new work unit...
[20:41:29] + Attempting to get work packet
[20:41:29] - Connecting to assignment server
[20:41:30] - Successful: assigned to (171.67.108.11).
[20:41:30] + News From Folding@Home: GPU folding beta
[20:41:30] Loaded queue successfully.
[20:41:31] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[20:41:38] + Attempting to get work packet
[20:41:38] - Connecting to assignment server
[20:41:39] - Successful: assigned to (171.67.108.11).
[20:41:39] + News From Folding@Home: GPU folding beta
[20:41:39] Loaded queue successfully.
[20:41:41] + Closed connections
[20:41:46] 
[20:41:46] + Processing work unit
[20:41:46] Core required: FahCore_11.exe
[20:41:46] Core found.
[20:41:46] Working on queue slot 00 [February 18 20:41:46 UTC]
[20:41:46] + Working ...
[20:41:47] 
[20:41:47] *------------------------------*
[20:41:47] Folding@Home GPU Core - Beta
[20:41:47] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[20:41:47] 
[20:41:47] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:41:47] Build host: amoeba
[20:41:47] Board Type: Nvidia
[20:41:47] Core      : 
[20:41:47] Preparing to commence simulation
[20:41:47] - Looking at optimizations...
[20:41:47] - Created dyn
[20:41:47] - Files status OK
[20:41:47] - Expanded 98791 -> 492276 (decompressed 498.3 percent)
[20:41:47] Called DecompressByteArray: compressed_data_size=98791 data_size=492276, decompressed_data_size=492276 diff=0
[20:41:47] - Digital signature verified
[20:41:47] 
[20:41:47] Project: 5756 (Run 1, Clone 269, Gen 95)
[20:41:47] 
[20:41:47] Assembly optimizations on if available.
[20:41:47] Entering M.D.
[20:41:54] Working on Protein
[20:42:01] Client config found, loading data.
[20:42:01] Starting GUI Server
[20:43:59] Completed 1%
[20:45:57] Completed 2%
[20:47:55] Completed 3%
[20:49:52] Completed 4%
[20:51:49] Completed 5%
[20:53:47] Completed 6%
[20:55:44] Completed 7%
[20:57:41] Completed 8%
[20:59:38] Completed 9%
[21:01:35] Completed 10%
[21:03:32] Completed 11%
[21:05:29] Completed 12%
[21:07:25] Completed 13%
[21:09:22] Completed 14%
[21:11:19] Completed 15%
[21:13:15] Completed 16%
[21:15:12] Completed 17%
[21:17:08] Completed 18%
[21:19:05] Completed 19%
[21:21:02] Completed 20%
[21:22:58] Completed 21%
[21:24:55] Completed 22%
[21:26:51] Completed 23%
[21:28:48] Completed 24%
[21:30:45] Completed 25%
[21:32:41] Completed 26%
[21:34:38] Completed 27%
[21:36:34] Completed 28%
[21:38:31] Completed 29%
[21:40:28] Completed 30%
[21:42:25] Completed 31%
[21:44:21] Completed 32%
[21:46:18] Completed 33%
[21:48:14] Completed 34%
[21:50:11] Completed 35%
[21:52:07] Completed 36%
[21:54:04] Completed 37%
[21:56:01] Completed 38%
[21:58:00] Completed 39%
[21:59:59] Completed 40%
[22:01:59] Completed 41%
[22:03:57] Completed 42%
[22:05:54] Completed 43%
[22:07:53] Completed 44%
[22:09:58] Completed 45%
[22:12:03] Completed 46%
[22:14:06] Completed 47%
[22:16:03] Completed 48%
[22:18:01] Completed 49%
[22:19:58] Completed 50%
[22:21:55] Completed 51%
[22:23:52] Completed 52%
[22:25:49] Completed 53%
[22:27:46] Completed 54%
[22:29:43] Completed 55%
[22:31:39] Completed 56%
[22:33:36] Completed 57%
[22:35:33] Completed 58%
[22:37:29] Completed 59%
[22:39:26] Completed 60%
[22:41:23] Completed 61%
[22:43:19] Completed 62%
[22:45:16] Completed 63%
[22:47:13] Completed 64%
[22:49:09] Completed 65%
[22:51:06] Completed 66%
[22:53:02] Completed 67%
[22:54:59] Completed 68%
[22:56:56] Completed 69%
[22:58:53] Completed 70%
[23:00:49] Completed 71%
[23:02:46] Completed 72%
[23:04:42] Completed 73%
[23:06:39] Completed 74%
[23:08:36] Completed 75%
[23:10:32] Completed 76%
[23:12:29] Completed 77%
[23:14:25] Completed 78%
[23:16:22] Completed 79%
[23:18:18] Completed 80%
[23:20:15] Completed 81%
[23:22:12] Completed 82%
[23:24:08] Completed 83%
[23:26:05] Completed 84%
[23:28:02] Completed 85%
[23:29:58] Completed 86%
[23:31:55] Completed 87%
[23:33:51] Completed 88%
[23:35:48] Completed 89%
[23:37:45] Completed 90%
[23:39:41] Completed 91%
[23:41:38] Completed 92%
[23:43:35] Completed 93%
[23:45:31] Completed 94%
[23:47:28] Completed 95%
[23:49:24] Completed 96%
[23:51:21] Completed 97%
[23:53:18] Completed 98%
[23:55:14] Completed 99%
[23:57:11] Completed 100%
[23:57:12] Successful run
[23:57:12] DynamicWrapper: Finished Work Unit: sleep=10000
[23:57:22] Reserved 111964 bytes for xtc file; Cosm status=0
[23:57:22] Allocated 111964 bytes for xtc file
[23:57:22] - Reading up to 111964 from "work/wudata_00.xtc": Read 111964
[23:57:22] Read 111964 bytes from xtc file; available packet space=786318500
[23:57:22] xtc file hash check passed.
[23:57:22] Reserved 33528 33528 786318500 bytes for arc file=<work/wudata_00.trr> Cosm status=0
[23:57:22] Allocated 33528 bytes for arc file
[23:57:22] - Reading up to 33528 from "work/wudata_00.trr": Read 33528
[23:57:22] Read 33528 bytes from arc file; available packet space=786284972
[23:57:22] trr file hash check passed.
[23:57:22] Allocated 560 bytes for edr file
[23:57:22] Read bedfile
[23:57:22] edr file hash check passed.
[23:57:22] Allocated 12540 bytes for logfile
[23:57:22] Read logfile
[23:57:22] GuardedRun: success in DynamicWrapper
[23:57:22] GuardedRun: done
[23:57:22] Run: GuardedRun completed.
[23:57:27] - Writing 159104 bytes of core data to disk...
[23:57:27] Done: 158592 -> 151434 (compressed to 95.4 percent)
[23:57:27]   ... Done.
[23:57:27] - Shutting down core 
[23:57:27] 
[23:57:27] Folding@home Core Shutdown: FINISHED_UNIT
[23:57:31] CoreStatus = 64 (100)
[23:57:31] Sending work to server
[23:57:31] Project: 5756 (Run 1, Clone 269, Gen 95)


[23:57:31] + Attempting to send results [February 18 23:57:31 UTC]
[23:57:38] + Results successfully sent
[23:57:38] Thank you for your contribution to Folding@Home.
[23:57:38] + Number of Units Completed: 14

[23:57:42] - Preparing to get new work unit...
[23:57:42] + Attempting to get work packet
[23:57:42] - Connecting to assignment server
[23:57:42] + Could not connect to Assignment Server
[23:57:42] + Could not connect to Assignment Server 2
[23:57:42] + Couldn't get work instructions.
[23:57:42] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[23:57:52] + Attempting to get work packet
[23:57:52] - Connecting to assignment server
[23:57:53] - Successful: assigned to (171.67.108.11).
[23:57:53] + News From Folding@Home: GPU folding beta
[23:57:53] Loaded queue successfully.
[23:57:55] + Closed connections
[23:57:55] 
[23:57:55] + Processing work unit
[23:57:55] Core required: FahCore_11.exe
[23:57:55] Core found.
[23:57:55] Working on queue slot 01 [February 18 23:57:55 UTC]
[23:57:55] + Working ...
[23:57:55] 
[23:57:55] *------------------------------*
[23:57:55] Folding@Home GPU Core - Beta
[23:57:55] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[23:57:55] 
[23:57:55] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[23:57:55] Build host: amoeba
[23:57:55] Board Type: Nvidia
[23:57:55] Core      : 
[23:57:55] Preparing to commence simulation
[23:57:55] - Looking at optimizations...
[23:57:55] - Created dyn
[23:57:55] - Files status OK
[23:57:55] - Expanded 96573 -> 489240 (decompressed 506.6 percent)
[23:57:55] Called DecompressByteArray: compressed_data_size=96573 data_size=489240, decompressed_data_size=489240 diff=0
[23:57:55] - Digital signature verified
[23:57:55] 
[23:57:55] Project: 5752 (Run 2, Clone 37, Gen 104)
[23:57:55] 
[23:57:55] Assembly optimizations on if available.
[23:57:55] Entering M.D.
[23:58:02] Working on Protein
[23:58:08] Client config found, loading data.
[23:58:08] Starting GUI Server
[00:00:05] Completed 1%
[00:02:01] Completed 2%
[00:03:57] Completed 3%
[00:05:53] Completed 4%
[00:07:50] Completed 5%
[00:09:46] Completed 6%
[00:11:42] Completed 7%
[00:13:39] Completed 8%
[00:15:35] Completed 9%
[00:17:31] Completed 10%
[00:19:28] Completed 11%
[00:21:24] Completed 12%
[00:23:20] Completed 13%
[00:25:16] Completed 14%
[00:27:12] Completed 15%
[00:29:09] Completed 16%
[00:31:05] Completed 17%
[00:33:02] Completed 18%
[00:34:58] Completed 19%
[00:36:54] Completed 20%
[00:38:50] Completed 21%
[00:40:47] Completed 22%
[00:42:43] Completed 23%
[00:44:39] Completed 24%
[00:46:35] Completed 25%
[00:48:32] Completed 26%
[00:50:28] Completed 27%
[00:52:24] Completed 28%
[00:54:21] Completed 29%
[00:56:17] Completed 30%
[00:58:13] Completed 31%
[01:00:09] Completed 32%
[01:02:06] Completed 33%
[01:04:02] Completed 34%
[01:06:00] Completed 35%
[01:07:56] Completed 36%
[01:09:53] Completed 37%
[01:11:49] Completed 38%
[01:13:45] Completed 39%
[01:15:41] Completed 40%
[01:17:38] Completed 41%
[01:19:34] Completed 42%
[01:21:30] Completed 43%
[01:23:27] Completed 44%
[01:25:23] Completed 45%
[01:27:19] Completed 46%
[01:29:16] Completed 47%
[01:31:12] Completed 48%
[01:33:08] Completed 49%
[01:35:04] Completed 50%
[01:37:01] Completed 51%
[01:38:57] Completed 52%
[01:40:53] Completed 53%
[01:42:49] Completed 54%
[01:44:46] Completed 55%
[01:46:42] Completed 56%
[01:48:38] Completed 57%
[01:50:35] Completed 58%
[01:52:31] Completed 59%
[01:54:27] Completed 60%
[01:56:23] Completed 61%
[01:58:19] Completed 62%
[02:00:16] Completed 63%
[02:02:12] Completed 64%
[02:04:08] Completed 65%
[02:06:05] Completed 66%
[02:08:01] Completed 67%
[02:09:57] Completed 68%
[02:11:53] Completed 69%
[02:13:50] Completed 70%
[02:15:46] Completed 71%
[02:17:42] Completed 72%
[02:19:39] Completed 73%
[02:21:35] Completed 74%
[02:23:31] Completed 75%
[02:25:27] Completed 76%
[02:27:24] Completed 77%
[02:29:20] Completed 78%
[02:31:16] Completed 79%
[02:33:13] Completed 80%
[02:35:09] Completed 81%
[02:37:05] Completed 82%
[02:39:01] Completed 83%
[02:40:58] Completed 84%
[02:41:11] + Working...
[02:42:54] Completed 85%
[02:44:50] Completed 86%
[02:46:47] Completed 87%
[02:48:43] Completed 88%
[02:50:39] Completed 89%
[02:52:36] Completed 90%
[02:54:31] Completed 91%
[02:56:28] Completed 92%
[02:58:24] Completed 93%
[03:00:20] Completed 94%
[03:02:17] Completed 95%
[03:04:13] Completed 96%
[03:06:09] Completed 97%
[03:08:06] Completed 98%
[03:10:02] Completed 99%
[03:11:58] Completed 100%
[03:12:00] Successful run
[03:12:00] DynamicWrapper: Finished Work Unit: sleep=10000
[03:12:09] Reserved 112060 bytes for xtc file; Cosm status=0
[03:12:09] Allocated 112060 bytes for xtc file
[03:12:09] - Reading up to 112060 from "work/wudata_01.xtc": Read 112060
[03:12:09] Read 112060 bytes from xtc file; available packet space=786318404
[03:12:09] xtc file hash check passed.
[03:12:09] Reserved 33528 33528 786318404 bytes for arc file=<work/wudata_01.trr> Cosm status=0
[03:12:09] Allocated 33528 bytes for arc file
[03:12:09] - Reading up to 33528 from "work/wudata_01.trr": Read 33528
[03:12:09] Read 33528 bytes from arc file; available packet space=786284876
[03:12:09] trr file hash check passed.
[03:12:09] Allocated 560 bytes for edr file
[03:12:09] Read bedfile
[03:12:09] edr file hash check passed.
[03:12:09] Allocated 13514 bytes for logfile
[03:12:09] Read logfile
[03:12:09] GuardedRun: success in DynamicWrapper
[03:12:09] GuardedRun: done
[03:12:09] Run: GuardedRun completed.
[03:12:10] - Writing 160174 bytes of core data to disk...
[03:12:10] Done: 159662 -> 151699 (compressed to 95.0 percent)
[03:12:10]   ... Done.
[03:12:10] - Shutting down core 
[03:12:10] 
[03:12:10] Folding@home Core Shutdown: FINISHED_UNIT
[03:12:15] CoreStatus = 64 (100)
[03:12:15] Sending work to server
[03:12:15] Project: 5752 (Run 2, Clone 37, Gen 104)


[03:12:15] + Attempting to send results [February 19 03:12:15 UTC]
[03:12:23] + Results successfully sent
[03:12:23] Thank you for your contribution to Folding@Home.
[03:12:23] + Number of Units Completed: 15

[03:12:27] - Preparing to get new work unit...
[03:12:27] + Attempting to get work packet
[03:12:27] - Connecting to assignment server
[03:12:27] - Successful: assigned to (171.67.108.11).
[03:12:27] + News From Folding@Home: GPU folding beta
[03:12:28] Loaded queue successfully.
[03:12:30] + Closed connections
[03:12:30] 
[03:12:30] + Processing work unit
[03:12:30] Core required: FahCore_11.exe
[03:12:30] Core found.
[03:12:30] Working on queue slot 02 [February 19 03:12:30 UTC]
[03:12:30] + Working ...
[03:12:30] 
[03:12:30] *------------------------------*
[03:12:30] Folding@Home GPU Core - Beta
[03:12:30] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[03:12:30] 
[03:12:30] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[03:12:30] Build host: amoeba
[03:12:30] Board Type: Nvidia
[03:12:30] Core      : 
[03:12:30] Preparing to commence simulation
[03:12:30] - Looking at optimizations...
[03:12:30] - Created dyn
[03:12:30] - Files status OK
[03:12:30] - Expanded 98723 -> 492276 (decompressed 498.6 percent)
[03:12:30] Called DecompressByteArray: compressed_data_size=98723 data_size=492276, decompressed_data_size=492276 diff=0
[03:12:30] - Digital signature verified
[03:12:30] 
[03:12:30] Project: 5749 (Run 11, Clone 390, Gen 97)
[03:12:30] 
[03:12:30] Assembly optimizations on if available.
[03:12:30] Entering M.D.
[03:12:37] Working on Protein
[03:12:43] Client config found, loading data.
[03:12:43] Starting GUI Server
[03:14:40] Completed 1%
[03:16:37] Completed 2%
[03:18:33] Completed 3%
[03:20:30] Completed 4%
[03:22:09] Run: exception thrown during GuardedRun
[03:22:09] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[03:22:09] Going to send back what have done -- stepsTotalG=10000000
[03:22:09] Work fraction=0.0485 steps=10000000.
[03:22:13] logfile size=0 infoLength=0 edr=0 trr=23
[03:22:13] - Writing 642 bytes of core data to disk...
[03:22:13] Done: 130 -> 127 (compressed to 97.6 percent)
[03:22:13]   ... Done.
[03:22:19] 
[03:22:19] Folding@home Core Shutdown: UNSTABLE_MACHINE
[03:22:22] CoreStatus = 7A (122)
[03:22:22] Sending work to server
[03:22:22] Project: 5749 (Run 11, Clone 390, Gen 97)


[03:22:22] + Attempting to send results [February 19 03:22:22 UTC]
[03:22:23] + Results successfully sent
[03:22:23] Thank you for your contribution to Folding@Home.
[03:22:27] - Preparing to get new work unit...
[03:22:27] + Attempting to get work packet
[03:22:27] - Connecting to assignment server
[03:22:27] + Could not connect to Assignment Server
[03:22:27] + Could not connect to Assignment Server 2
[03:22:27] + Couldn't get work instructions.
[03:22:27] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[03:22:32] + Attempting to get work packet
[03:22:32] - Connecting to assignment server
[03:22:33] - Successful: assigned to (171.67.108.11).
[03:22:33] + News From Folding@Home: GPU folding beta
[03:22:33] Loaded queue successfully.
[03:22:35] + Closed connections
[03:22:40] 
[03:22:40] + Processing work unit
[03:22:40] Core required: FahCore_11.exe
[03:22:40] Core found.
[03:22:40] Working on queue slot 03 [February 19 03:22:40 UTC]
[03:22:40] + Working ...
[03:22:40] 
[03:22:40] *------------------------------*
[03:22:40] Folding@Home GPU Core - Beta
[03:22:40] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[03:22:40] 
[03:22:40] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[03:22:40] Build host: amoeba
[03:22:40] Board Type: Nvidia
[03:22:40] Core      : 
[03:22:40] Preparing to commence simulation
[03:22:40] - Looking at optimizations...
[03:22:40] - Created dyn
[03:22:40] - Files status OK
[03:22:40] - Expanded 46738 -> 252912 (decompressed 541.1 percent)
[03:22:40] Called DecompressByteArray: compressed_data_size=46738 data_size=252912, decompressed_data_size=252912 diff=0
[03:22:40] - Digital signature verified
[03:22:40] 
[03:22:40] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:22:40] 
[03:22:40] Assembly optimizations on if available.
[03:22:40] Entering M.D.
[03:22:46] Working on Protein
[03:22:48] Client config found, loading data.
[03:22:48] mdrun_Starting GUI Server
[03:22:48] gpu returned 
[03:22:48] NANs detected on GPU
[03:22:48] 
[03:22:48] Folding@home Core Shutdown: UNSTABLE_MACHINE
[03:22:52] CoreStatus = 7A (122)
[03:22:52] Sending work to server
[03:22:52] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:22:52] - Error: Could not get length of results file work/wuresults_03.dat
[03:22:52] - Error: Could not read unit 03 file. Removing from queue.
[03:22:52] - Preparing to get new work unit...
[03:22:52] + Attempting to get work packet
[03:22:52] - Connecting to assignment server
[03:22:53] - Successful: assigned to (171.67.108.11).
[03:22:53] + News From Folding@Home: GPU folding beta
[03:22:53] Loaded queue successfully.
[03:22:55] + Closed connections
[03:23:00] 
[03:23:00] + Processing work unit
[03:23:00] Core required: FahCore_11.exe
[03:23:00] Core found.
[03:23:00] Working on queue slot 04 [February 19 03:23:00 UTC]
[03:23:00] + Working ...
[03:23:00] 
[03:23:00] *------------------------------*
[03:23:00] Folding@Home GPU Core - Beta
[03:23:00] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[03:23:00] 
[03:23:00] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[03:23:00] Build host: amoeba
[03:23:00] Board Type: Nvidia
[03:23:00] Core      : 
[03:23:00] Preparing to commence simulation
[03:23:00] - Looking at optimizations...
[03:23:00] - Created dyn
[03:23:00] - Files status OK
[03:23:00] - Expanded 46738 -> 252912 (decompressed 541.1 percent)
[03:23:00] Called DecompressByteArray: compressed_data_size=46738 data_size=252912, decompressed_data_size=252912 diff=0
[03:23:00] - Digital signature verified
[03:23:00] 
[03:23:00] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:00] 
[03:23:00] Assembly optimizations on if available.
[03:23:00] Entering M.D.
[03:23:06] Working on Protein
[03:23:08] Client config found, loading data.
[03:23:08] mdrun_gpu returned 
[03:23:08] NANs detected on GPU
[03:23:08] 
[03:23:08] Folding@home Core Shutdown: UNSTABLE_MACHINE
[03:23:12] CoreStatus = 7A (122)
[03:23:12] Sending work to server
[03:23:12] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:12] - Error: Could not get length of results file work/wuresults_04.dat
[03:23:12] - Error: Could not read unit 04 file. Removing from queue.
[03:23:12] - Preparing to get new work unit...
[03:23:12] + Attempting to get work packet
[03:23:12] - Connecting to assignment server
[03:23:13] - Successful: assigned to (171.67.108.11).
[03:23:13] + News From Folding@Home: GPU folding beta
[03:23:13] Loaded queue successfully.
[03:23:15] + Closed connections
[03:23:20] 
[03:23:20] + Processing work unit
[03:23:20] Core required: FahCore_11.exe
[03:23:20] Core found.
[03:23:20] Working on queue slot 05 [February 19 03:23:20 UTC]
[03:23:20] + Working ...
[03:23:20] 
[03:23:20] *------------------------------*
[03:23:20] Folding@Home GPU Core - Beta
[03:23:20] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[03:23:20] 
[03:23:20] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[03:23:20] Build host: amoeba
[03:23:20] Board Type: Nvidia
[03:23:20] Core      : 
[03:23:20] Preparing to commence simulation
[03:23:20] - Looking at optimizations...
[03:23:20] - Created dyn
[03:23:20] - Files status OK
[03:23:20] - Expanded 46738 -> 252912 (decompressed 541.1 percent)
[03:23:20] Called DecompressByteArray: compressed_data_size=46738 data_size=252912, decompressed_data_size=252912 diff=0
[03:23:20] - Digital signature verified
[03:23:20] 
[03:23:20] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:20] 
[03:23:20] Assembly optimizations on if available.
[03:23:20] Entering M.D.
[03:23:26] Working on Protein
[03:23:28] Client config found, loading data.
[03:23:28] Starting GUI Server
[03:23:28] mdrun_gpu returned 
[03:23:28] NANs detected on GPU
[03:23:28] 
[03:23:28] Folding@home Core Shutdown: UNSTABLE_MACHINE
[03:23:32] CoreStatus = 7A (122)
[03:23:32] Sending work to server
[03:23:32] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:32] - Error: Could not get length of results file work/wuresults_05.dat
[03:23:32] - Error: Could not read unit 05 file. Removing from queue.
[03:23:32] - Preparing to get new work unit...
[03:23:32] + Attempting to get work packet
[03:23:32] - Connecting to assignment server
[03:23:33] - Successful: assigned to (171.67.108.11).
[03:23:33] + News From Folding@Home: GPU folding beta
[03:23:33] Loaded queue successfully.
[03:23:34] + Closed connections
[03:23:39] 
[03:23:39] + Processing work unit
[03:23:39] Core required: FahCore_11.exe
[03:23:39] Core found.
[03:23:39] Working on queue slot 06 [February 19 03:23:39 UTC]
[03:23:39] + Working ...
[03:23:39] 
[03:23:39] *------------------------------*
[03:23:39] Folding@Home GPU Core - Beta
[03:23:39] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[03:23:39] 
[03:23:39] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[03:23:39] Build host: amoeba
[03:23:39] Board Type: Nvidia
[03:23:39] Core      : 
[03:23:39] Preparing to commence simulation
[03:23:39] - Looking at optimizations...
[03:23:39] - Created dyn
[03:23:39] - Files status OK
[03:23:39] - Expanded 46738 -> 252912 (decompressed 541.1 percent)
[03:23:39] Called DecompressByteArray: compressed_data_size=46738 data_size=252912, decompressed_data_size=252912 diff=0
[03:23:40] - Digital signature verified
[03:23:40] 
[03:23:40] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:40] 
[03:23:40] Assembly optimizations on if available.
[03:23:40] Entering M.D.
[03:23:46] Working on Protein
[03:23:47] Client config found, loading data.
[03:23:47] Starting GUI Server
[03:23:47] mdrun_gpu returned 
[03:23:47] NANs detected on GPU
[03:23:47] 
[03:23:47] Folding@home Core Shutdown: UNSTABLE_MACHINE
[03:23:52] CoreStatus = 7A (122)
[03:23:52] Sending work to server
[03:23:52] Project: 5765 (Run 2, Clone 106, Gen 40)
[03:23:52] - Error: Could not get length of results file work/wuresults_06.dat
[03:23:52] - Error: Could not read unit 06 file. Removing from queue.
[03:23:52] EUE limit exceeded. Pausing 24 hours.

Re: 5765 NANs despite data del, reinstall, latest NV drv

Posted: Thu Feb 19, 2009 12:38 pm
by toTOW
Did you see these threads :

viewtopic.php?f=52&t=7953
viewtopic.php?f=52&t=7965

Re: 5765 NANs despite data del, reinstall, latest NV drv

Posted: Thu Feb 19, 2009 2:30 pm
by gue22
>Did you see these threads?<
Yes, I read up on this apparently big issue be4 I posted.

Here´s my run through the added log in the original post 4 ur convenience:
5765 (Run 7, Clone 120, Gen 188) repeatedly fails.
Then
5756 (Run 1, Clone 269, Gen 95) and
5752 (Run 2, Clone 37, Gen 104) run nicely.

5749 (Run 11, Clone 390, Gen 97) fails after 4% w/ CoreStatus = 7A
5765 (Run 2, Clone 106, Gen 40) stopped after repeated fails.

From what I can tell it´s not my machine and I´ll be back @ Stanford after you resolved the issue.
Or is there anything I overlooked or else I can do?
Greetings
G.

Re: 5765 NANs despite data del, reinstall, latest NV drv

Posted: Thu Feb 19, 2009 7:40 pm
by gue22
With their share of problems at the other side of the Bay <grin> I gave FaH another shot.

Bottom line:
5771 (Run 2, Clone 225, Gen 102) worked fine.
5765 (Run 0, Clone 424, Gen 198) - after several failed attempts:
EUE limit exceeded. Pausing 24 hours.

That´s exactly what I intend to do. <g>
G.

Code: Select all

--- Opening Log file [February 19 16:58:16 UTC] 


# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\gy\AppData\Roaming\Folding@home-gpu


[16:58:16] - Ask before connecting: No
[16:58:16] - User name: gue22 (Team 0)
[16:58:16] - User ID: 364F170142FAF600
[16:58:16] - Machine ID: 2
[16:58:16] 
[16:58:16] Loaded queue successfully.
[16:58:16] Initialization complete
[16:58:16] - Preparing to get new work unit...
[16:58:16] + Attempting to get work packet
[16:58:16] - Connecting to assignment server
[16:58:16] + Could not connect to Assignment Server
[16:58:17] - Successful: assigned to (171.67.108.11).
[16:58:17] + News From Folding@Home: GPU folding beta
[16:58:17] Loaded queue successfully.
[16:58:19] + Closed connections
[16:58:19] 
[16:58:19] + Processing work unit
[16:58:19] Core required: FahCore_11.exe
[16:58:19] Core found.
[16:58:19] Working on queue slot 07 [February 19 16:58:19 UTC]
[16:58:19] + Working ...
[16:58:19] 
[16:58:19] *------------------------------*
[16:58:19] Folding@Home GPU Core - Beta
[16:58:19] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:58:19] 
[16:58:19] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:58:19] Build host: amoeba
[16:58:19] Board Type: Nvidia
[16:58:19] Core      : 
[16:58:19] Preparing to commence simulation
[16:58:19] - Looking at optimizations...
[16:58:19] - Created dyn
[16:58:19] - Files status OK
[16:58:19] - Expanded 45412 -> 251112 (decompressed 552.9 percent)
[16:58:19] Called DecompressByteArray: compressed_data_size=45412 data_size=251112, decompressed_data_size=251112 diff=0
[16:58:19] - Digital signature verified
[16:58:19] 
[16:58:19] Project: 5771 (Run 2, Clone 225, Gen 102)
[16:58:19] 
[16:58:19] Assembly optimizations on if available.
[16:58:19] Entering M.D.
[16:58:25] Working on Protein
[16:58:27] Client config found, loading data.
[16:58:27] Starting GUI Server
[16:59:20] Completed 1%
[17:00:17] Completed 2%
[17:01:14] Completed 3%
[17:02:11] Completed 4%
[17:03:09] Completed 5%
[17:04:00] Completed 6%
[17:04:52] Completed 7%
[17:05:43] Completed 8%
[17:06:35] Completed 9%
[17:07:26] Completed 10%
[17:08:17] Completed 11%
[17:09:09] Completed 12%
[17:10:00] Completed 13%
[17:10:52] Completed 14%
[17:11:43] Completed 15%
[17:12:35] Completed 16%
[17:13:26] Completed 17%
[17:14:17] Completed 18%
[17:15:09] Completed 19%
[17:16:00] Completed 20%
[17:16:51] Completed 21%
[17:17:43] Completed 22%
[17:18:34] Completed 23%
[17:19:25] Completed 24%
[17:20:16] Completed 25%
[17:21:08] Completed 26%
[17:21:59] Completed 27%
[17:22:50] Completed 28%
[17:23:41] Completed 29%
[17:24:33] Completed 30%
[17:25:24] Completed 31%
[17:26:15] Completed 32%
[17:27:07] Completed 33%
[17:27:58] Completed 34%
[17:28:49] Completed 35%
[17:29:40] Completed 36%
[17:30:32] Completed 37%
[17:31:23] Completed 38%
[17:32:14] Completed 39%
[17:33:06] Completed 40%
[17:33:57] Completed 41%
[17:34:48] Completed 42%
[17:35:39] Completed 43%
[17:36:31] Completed 44%
[17:37:22] Completed 45%
[17:38:13] Completed 46%
[17:39:04] Completed 47%
[17:39:56] Completed 48%
[17:40:47] Completed 49%
[17:41:38] Completed 50%
[17:42:29] Completed 51%
[17:43:21] Completed 52%
[17:44:12] Completed 53%
[17:45:03] Completed 54%
[17:45:55] Completed 55%
[17:46:46] Completed 56%
[17:47:37] Completed 57%
[17:48:28] Completed 58%
[17:49:20] Completed 59%
[17:50:11] Completed 60%
[17:51:02] Completed 61%
[17:51:54] Completed 62%
[17:52:45] Completed 63%
[17:53:36] Completed 64%
[17:54:27] Completed 65%
[17:55:19] Completed 66%
[17:56:10] Completed 67%
[17:57:02] Completed 68%
[17:57:54] Completed 69%
[17:58:46] Completed 70%
[17:59:38] Completed 71%
[18:00:30] Completed 72%
[18:01:21] Completed 73%
[18:02:13] Completed 74%
[18:03:05] Completed 75%
[18:03:57] Completed 76%
[18:04:50] Completed 77%
[18:05:45] Completed 78%
[18:06:40] Completed 79%
[18:07:35] Completed 80%
[18:08:28] Completed 81%
[18:09:23] Completed 82%
[18:10:19] Completed 83%
[18:11:15] Completed 84%
[18:12:11] Completed 85%
[18:13:07] Completed 86%
[18:14:03] Completed 87%
[18:14:59] Completed 88%
[18:15:55] Completed 89%
[18:16:50] Completed 90%
[18:17:47] Completed 91%
[18:18:51] Completed 92%
[18:19:51] Completed 93%
[18:20:46] Completed 94%
[18:21:38] Completed 95%
[18:22:30] Completed 96%
[18:23:23] Completed 97%
[18:24:15] Completed 98%
[18:25:13] Completed 99%
[18:26:14] Completed 100%
[18:26:15] Successful run
[18:26:15] DynamicWrapper: Finished Work Unit: sleep=10000
[18:26:25] Reserved 75908 bytes for xtc file; Cosm status=0
[18:26:25] Allocated 75908 bytes for xtc file
[18:26:25] - Reading up to 75908 from "work/wudata_07.xtc": Read 75908
[18:26:25] Read 75908 bytes from xtc file; available packet space=786354556
[18:26:25] xtc file hash check passed.
[18:26:25] Reserved 15168 15168 786354556 bytes for arc file=<work/wudata_07.trr> Cosm status=0
[18:26:25] Allocated 15168 bytes for arc file
[18:26:25] - Reading up to 15168 from "work/wudata_07.trr": Read 15168
[18:26:25] Read 15168 bytes from arc file; available packet space=786339388
[18:26:25] trr file hash check passed.
[18:26:25] Allocated 560 bytes for edr file
[18:26:25] Read bedfile
[18:26:25] edr file hash check passed.
[18:26:25] Allocated 23306 bytes for logfile
[18:26:25] Read logfile
[18:26:25] GuardedRun: success in DynamicWrapper
[18:26:25] GuardedRun: done
[18:26:25] Run: GuardedRun completed.
[18:26:29] - Writing 115454 bytes of core data to disk...
[18:26:29] Done: 114942 -> 97542 (compressed to 84.8 percent)
[18:26:29]   ... Done.
[18:26:29] - Shutting down core 
[18:26:29] 
[18:26:29] Folding@home Core Shutdown: FINISHED_UNIT
[18:26:33] CoreStatus = 64 (100)
[18:26:33] Sending work to server
[18:26:33] Project: 5771 (Run 2, Clone 225, Gen 102)


[18:26:33] + Attempting to send results [February 19 18:26:33 UTC]
[18:26:40] + Results successfully sent
[18:26:40] Thank you for your contribution to Folding@Home.
[18:26:40] + Number of Units Completed: 16

[18:26:44] - Preparing to get new work unit...
[18:26:44] + Attempting to get work packet
[18:26:44] - Connecting to assignment server
[18:26:46] - Successful: assigned to (171.67.108.11).
[18:26:46] + News From Folding@Home: GPU folding beta
[18:26:47] Loaded queue successfully.
[18:26:50] + Closed connections
[18:26:50] 
[18:26:50] + Processing work unit
[18:26:50] Core required: FahCore_11.exe
[18:26:50] Core found.
[18:26:50] Working on queue slot 08 [February 19 18:26:50 UTC]
[18:26:50] + Working ...
[18:26:50] 
[18:26:50] *------------------------------*
[18:26:50] Folding@Home GPU Core - Beta
[18:26:50] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[18:26:50] 
[18:26:50] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[18:26:50] Build host: amoeba
[18:26:50] Board Type: Nvidia
[18:26:50] Core      : 
[18:26:50] Preparing to commence simulation
[18:26:50] - Looking at optimizations...
[18:26:50] - Created dyn
[18:26:50] - Files status OK
[18:26:51] - Expanded 46753 -> 252912 (decompressed 540.9 percent)
[18:26:51] Called DecompressByteArray: compressed_data_size=46753 data_size=252912, decompressed_data_size=252912 diff=0
[18:26:51] - Digital signature verified
[18:26:51] 
[18:26:51] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:26:51] 
[18:26:51] Assembly optimizations on if available.
[18:26:51] Entering M.D.
[18:26:58] Working on Protein
[18:27:00] Client config found, loading data.
[18:27:00] mdrun_gpu returned 
[18:27:00] NANs detected on GPU
[18:27:00] 
[18:27:00] Folding@home Core Shutdown: UNSTABLE_MACHINE
[18:27:05] CoreStatus = 7A (122)
[18:27:05] Sending work to server
[18:27:05] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:27:05] - Error: Could not get length of results file work/wuresults_08.dat
[18:27:05] - Error: Could not read unit 08 file. Removing from queue.
[18:27:05] - Preparing to get new work unit...
[18:27:05] + Attempting to get work packet
[18:27:05] - Connecting to assignment server
[18:27:07] - Successful: assigned to (171.67.108.11).
[18:27:07] + News From Folding@Home: GPU folding beta
[18:27:07] Loaded queue successfully.
[18:27:12] + Closed connections
[18:27:17] 
[18:27:17] + Processing work unit
[18:27:17] Core required: FahCore_11.exe
[18:27:17] Core found.
[18:27:17] Working on queue slot 09 [February 19 18:27:17 UTC]
[18:27:17] + Working ...
[18:27:17] 
[18:27:17] *------------------------------*
[18:27:17] Folding@Home GPU Core - Beta
[18:27:17] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[18:27:17] 
[18:27:17] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[18:27:17] Build host: amoeba
[18:27:17] Board Type: Nvidia
[18:27:17] Core      : 
[18:27:17] Preparing to commence simulation
[18:27:17] - Looking at optimizations...
[18:27:17] - Created dyn
[18:27:17] - Files status OK
[18:27:17] - Expanded 46753 -> 252912 (decompressed 540.9 percent)
[18:27:17] Called DecompressByteArray: compressed_data_size=46753 data_size=252912, decompressed_data_size=252912 diff=0
[18:27:17] - Digital signature verified
[18:27:17] 
[18:27:17] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:27:17] 
[18:27:18] Assembly optimizations on if available.
[18:27:18] Entering M.D.
[18:27:25] Working on Protein
[18:27:27] Client config found, loading data.
[18:27:27] mdrun_gpu returned 
[18:27:27] NANs detected on GPU
[18:27:27] 
[18:27:27] Folding@home Core Shutdown: UNSTABLE_MACHINE
[18:27:28] Starting GUI Server
[18:27:33] CoreStatus = 7A (122)
[18:27:33] Sending work to server
[18:27:33] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:27:33] - Error: Could not get length of results file work/wuresults_09.dat
[18:27:33] - Error: Could not read unit 09 file. Removing from queue.
[18:27:33] - Preparing to get new work unit...
[18:27:33] + Attempting to get work packet
[18:27:33] - Connecting to assignment server
[18:27:36] - Successful: assigned to (171.67.108.11).
[18:27:36] + News From Folding@Home: GPU folding beta
[18:27:36] Loaded queue successfully.
[18:27:42] + Closed connections
[18:27:47] 
[18:27:47] + Processing work unit
[18:27:47] Core required: FahCore_11.exe
[18:27:47] Core found.
[18:27:47] Working on queue slot 00 [February 19 18:27:47 UTC]
[18:27:47] + Working ...
[18:27:47] 
[18:27:47] *------------------------------*
[18:27:47] Folding@Home GPU Core - Beta
[18:27:47] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[18:27:47] 
[18:27:47] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[18:27:47] Build host: amoeba
[18:27:47] Board Type: Nvidia
[18:27:47] Core      : 
[18:27:47] Preparing to commence simulation
[18:27:47] - Looking at optimizations...
[18:27:47] - Created dyn
[18:27:47] - Files status OK
[18:27:47] - Expanded 46753 -> 252912 (decompressed 540.9 percent)
[18:27:47] Called DecompressByteArray: compressed_data_size=46753 data_size=252912, decompressed_data_size=252912 diff=0
[18:27:47] - Digital signature verified
[18:27:47] 
[18:27:47] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:27:47] 
[18:27:47] Assembly optimizations on if available.
[18:27:47] Entering M.D.
[18:27:53] Working on Protein
[18:27:56] Client config found, loading data.
[18:27:56] Starting GUI Server
[18:27:56] mdrun_gpu returned 
[18:27:56] NANs detected on GPU
[18:27:56] 
[18:27:56] Folding@home Core Shutdown: UNSTABLE_MACHINE
[18:28:01] CoreStatus = 7A (122)
[18:28:01] Sending work to server
[18:28:01] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:28:01] - Error: Could not get length of results file work/wuresults_00.dat
[18:28:01] - Error: Could not read unit 00 file. Removing from queue.
[18:28:01] - Preparing to get new work unit...
[18:28:01] + Attempting to get work packet
[18:28:01] - Connecting to assignment server
[18:28:04] - Successful: assigned to (171.67.108.11).
[18:28:04] + News From Folding@Home: GPU folding beta
[18:28:04] Loaded queue successfully.
[18:28:09] + Closed connections
[18:28:14] 
[18:28:14] + Processing work unit
[18:28:14] Core required: FahCore_11.exe
[18:28:14] Core found.
[18:28:14] Working on queue slot 01 [February 19 18:28:14 UTC]
[18:28:14] + Working ...
[18:28:15] 
[18:28:15] *------------------------------*
[18:28:15] Folding@Home GPU Core - Beta
[18:28:15] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[18:28:15] 
[18:28:15] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[18:28:15] Build host: amoeba
[18:28:15] Board Type: Nvidia
[18:28:15] Core      : 
[18:28:15] Preparing to commence simulation
[18:28:15] - Looking at optimizations...
[18:28:15] - Created dyn
[18:28:15] - Files status OK
[18:28:15] - Expanded 46753 -> 252912 (decompressed 540.9 percent)
[18:28:15] Called DecompressByteArray: compressed_data_size=46753 data_size=252912, decompressed_data_size=252912 diff=0
[18:28:15] - Digital signature verified
[18:28:15] 
[18:28:15] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:28:15] 
[18:28:15] Assembly optimizations on if available.
[18:28:15] Entering M.D.
[18:28:21] Working on Protein
[18:28:23] Client config found, loading data.
[18:28:23] mdrun_gpu returned 
[18:28:23] NANs detected on GPU
[18:28:23] 
[18:28:23] Folding@home Core Shutdown: UNSTABLE_MACHINE
[18:28:29] CoreStatus = 7A (122)
[18:28:29] Sending work to server
[18:28:29] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:28:29] - Error: Could not get length of results file work/wuresults_01.dat
[18:28:29] - Error: Could not read unit 01 file. Removing from queue.
[18:28:29] - Preparing to get new work unit...
[18:28:29] + Attempting to get work packet
[18:28:29] - Connecting to assignment server
[18:28:31] - Successful: assigned to (171.67.108.11).
[18:28:31] + News From Folding@Home: GPU folding beta
[18:28:31] Loaded queue successfully.
[18:28:37] + Closed connections
[18:28:42] 
[18:28:42] + Processing work unit
[18:28:42] Core required: FahCore_11.exe
[18:28:42] Core found.
[18:28:42] Working on queue slot 02 [February 19 18:28:42 UTC]
[18:28:42] + Working ...
[18:28:42] 
[18:28:42] *------------------------------*
[18:28:42] Folding@Home GPU Core - Beta
[18:28:42] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[18:28:42] 
[18:28:42] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[18:28:42] Build host: amoeba
[18:28:42] Board Type: Nvidia
[18:28:42] Core      : 
[18:28:42] Preparing to commence simulation
[18:28:42] - Looking at optimizations...
[18:28:42] - Created dyn
[18:28:42] - Files status OK
[18:28:42] - Expanded 46753 -> 252912 (decompressed 540.9 percent)
[18:28:42] Called DecompressByteArray: compressed_data_size=46753 data_size=252912, decompressed_data_size=252912 diff=0
[18:28:42] - Digital signature verified
[18:28:42] 
[18:28:42] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:28:42] 
[18:28:42] Assembly optimizations on if available.
[18:28:42] Entering M.D.
[18:28:49] Working on Protein
[18:28:51] Client config found, loading data.
[18:28:51] Starting GUI Server
[18:28:51] mdrun_gpu returned 
[18:28:51] NANs detected on GPU
[18:28:51] 
[18:28:51] Folding@home Core Shutdown: UNSTABLE_MACHINE
[18:28:56] CoreStatus = 7A (122)
[18:28:56] Sending work to server
[18:28:56] Project: 5765 (Run 0, Clone 424, Gen 198)
[18:28:56] - Error: Could not get length of results file work/wuresults_02.dat
[18:28:56] - Error: Could not read unit 02 file. Removing from queue.
[18:28:56] EUE limit exceeded. Pausing 24 hours.

Re: 5765 NANs despite data del, reinstall, latest NV drv

Posted: Mon Feb 23, 2009 10:23 pm
by gue22
Today´s collection of failures:
5767 (Run 12, Clone 17, Gen 219)
5766 (Run 11, Clone 113, Gen 217)

Code: Select all

--- Opening Log file [February 23 09:00:57 UTC] 


# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\gy\AppData\Roaming\Folding@home-gpu


[09:00:57] - Ask before connecting: No
[09:00:57] - User name: gue22 (Team 0)
[09:00:57] - User ID: 364F170142FAF600
[09:00:57] - Machine ID: 2
[09:00:57] 
[09:00:57] Loaded queue successfully.
[09:00:57] Initialization complete
[09:00:57] - Preparing to get new work unit...
[09:00:57] + Attempting to get work packet
[09:00:57] - Connecting to assignment server
[09:00:57] + Could not connect to Assignment Server
[09:00:57] + Could not connect to Assignment Server 2
[09:00:57] + Couldn't get work instructions.
[09:00:57] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[09:01:12] + Attempting to get work packet
[09:01:12] - Connecting to assignment server
[09:01:12] + Could not connect to Assignment Server
[09:01:12] + Could not connect to Assignment Server 2
[09:01:12] + Couldn't get work instructions.
[09:01:12] - Attempt #2  to get work failed, and no other work to do.
Waiting before retry.
[09:01:23] + Attempting to get work packet
[09:01:23] - Connecting to assignment server
[09:01:24] - Successful: assigned to (171.67.108.11).
[09:01:24] + News From Folding@Home: GPU folding beta
[09:01:24] Loaded queue successfully.
[09:01:26] + Closed connections
[09:01:26] 
[09:01:26] + Processing work unit
[09:01:26] Core required: FahCore_11.exe
[09:01:26] Core found.
[09:01:26] Working on queue slot 06 [February 23 09:01:26 UTC]
[09:01:26] + Working ...
[09:01:26] 
[09:01:26] *------------------------------*
[09:01:26] Folding@Home GPU Core - Beta
[09:01:26] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:01:26] 
[09:01:26] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:01:26] Build host: amoeba
[09:01:26] Board Type: Nvidia
[09:01:26] Core      : 
[09:01:26] Preparing to commence simulation
[09:01:26] - Looking at optimizations...
[09:01:26] - Created dyn
[09:01:26] - Files status OK
[09:01:26] - Expanded 46757 -> 252912 (decompressed 540.9 percent)
[09:01:26] Called DecompressByteArray: compressed_data_size=46757 data_size=252912, decompressed_data_size=252912 diff=0
[09:01:26] - Digital signature verified
[09:01:26] 
[09:01:26] Project: 5766 (Run 11, Clone 113, Gen 217)
[09:01:26] 
[09:01:26] Assembly optimizations on if available.
[09:01:26] Entering M.D.
[09:01:32] Working on Protein
[09:01:33] Client config found, loading data.
[09:01:33] mdrun_gpu returned 
[09:01:33] NANs detected on GPU
[09:01:33] 
[09:01:33] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:01:36] CoreStatus = 7A (122)
[09:01:36] Sending work to server
[09:01:36] Project: 5766 (Run 11, Clone 113, Gen 217)
[09:01:36] - Error: Could not get length of results file work/wuresults_06.dat
[09:01:36] - Error: Could not read unit 06 file. Removing from queue.
[09:01:36] - Preparing to get new work unit...
[09:01:36] + Attempting to get work packet
[09:01:36] - Connecting to assignment server
[09:01:37] - Successful: assigned to (171.67.108.11).
[09:01:37] + News From Folding@Home: GPU folding beta
[09:01:37] Loaded queue successfully.
[09:01:39] + Closed connections

Folding@Home Client Shutdown.