7626 run:11 clone:0 gen:148

Moderators: Site Moderators, FAHC Science Team

Post Reply
wrinvert
Posts: 16
Joined: Sat Oct 22, 2011 8:24 am

7626 run:11 clone:0 gen:148

Post by wrinvert »

this one error 5 time befor I caught it, haven't had any issue with the 762X until this one with no changes to the system.

Code: Select all

*********************** Log Started 2013-05-17T12:01:31Z ***********************
12:01:31:************************* Folding@home Client *************************
12:01:31:      Website: http://folding.stanford.edu/
12:01:31:    Copyright: (c) 2009-2012 Stanford University
12:01:31:       Author: Joseph Coffland <[email protected]>
12:01:31:         Args: --lifeline 2184 --command-port=36330
12:01:31:       Config: E:/V7 folding/config.xml
12:01:31:******************************** Build ********************************
12:01:31:      Version: 7.2.9
12:01:31:         Date: Oct 3 2012
12:01:31:         Time: 18:05:48
12:01:31:      SVN Rev: 3578
12:01:31:       Branch: fah/trunk/client
12:01:31:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
12:01:31:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
12:01:31:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
12:01:31:     Platform: win32 XP
12:01:31:         Bits: 32
12:01:31:         Mode: Release
12:01:31:******************************* System ********************************
12:01:31:          CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
12:01:31:       CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
12:01:31:         CPUs: 12
12:01:31:       Memory: 15.95GiB
12:01:31:  Free Memory: 13.97GiB
12:01:31:      Threads: WINDOWS_THREADS
12:01:31:   On Battery: false
12:01:31:   UTC offset: -4
12:01:31:          PID: 3800
12:01:31:          CWD: E:/V7 folding
12:01:31:           OS: Windows 7 Home Premium
12:01:31:      OS Arch: AMD64
12:01:31:         GPUs: 3
12:01:31:        GPU 0: NVIDIA:2 GF114 [GeForce GTX 680]
12:01:31:        GPU 1: NVIDIA:2 GF114 [GeForce GTX 680]
12:01:31:        GPU 2: NVIDIA:2 GF114 [GeForce GTX 680]
12:01:31:         CUDA: 3.0
12:01:31:  CUDA Driver: 5050
12:01:31:Win32 Service: false
12:01:31:***********************************************************************
12:01:31:<config>
12:01:31:  <!-- FahCore Control -->
12:01:31:  <core-priority v='low'/>
12:01:31:
12:01:31:  <!-- Network -->
12:01:31:  <proxy v=':8080'/>
12:01:31:
12:01:31:  <!-- Remote Command Server -->
12:01:31:  <command-allow v='127.0.0.1,192.168.1.102/104/106'/>
12:01:31:  <password v='**********'/>
12:01:31:
12:01:31:  <!-- Slot Control -->
12:01:31:  <pause-on-start v='true'/>
12:01:31:
12:01:31:  <!-- User Information -->
12:01:31:  <passkey v='********************************'/>
12:01:31:  <team v='111065'/>
12:01:31:  <user v='wrinvert'/>
12:01:31:
12:01:31:  <!-- Folding Slots -->
12:01:31:  <slot id='0' type='SMP'>
12:01:31:    <cpus v='8'/>
12:01:31:  </slot>
12:01:31:  <slot id='1' type='GPU'>
12:01:31:    <client-type v='beta'/>
12:01:31:    <cuda-index v='0'/>
12:01:31:    <opencl-index v='0'/>
12:01:31:  </slot>
12:01:31:  <slot id='2' type='GPU'>
12:01:31:    <client-type v='beta'/>
12:01:31:    <cuda-index v='2'/>
12:01:31:    <opencl-index v='2'/>
12:01:31:  </slot>
12:01:31:  <slot id='3' type='GPU'>
12:01:31:    <client-type v='beta'/>
12:01:31:    <cuda-index v='1'/>
12:01:31:    <opencl-index v='1'/>
12:01:31:  </slot>
12:01:31:</config>
******************************** Date: 18/05/13 ********************************
05:01:12:WU03:FS03:Connecting to assign-GPU.stanford.edu:80
05:01:12:WU03:FS03:News: Welcome to Folding@Home
05:01:12:WU03:FS03:Assigned to work server 171.64.65.105
05:01:12:WU03:FS03:Requesting new work unit for slot 03: RUNNING gpu:2:"GF114 [GeForce GTX 680]" from 171.64.65.105
05:01:12:WU03:FS03:Connecting to 171.64.65.105:8080
05:01:13:WU03:FS03:Downloading 124.31KiB
05:01:14:WU03:FS03:Download complete
05:01:14:WU03:FS03:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:7626 run:11 clone:0 gen:148 core:0x15 unit:0x000000dd664f2dd14fe619ce9b068f27
05:01:14:WU03:FS03:Starting
05:01:14:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
05:01:14:WU03:FS03:Started FahCore on PID 2788
05:01:14:WU03:FS03:Core PID:3128
05:01:14:WU03:FS03:FahCore 0x15 started
05:01:15:WU03:FS03:0x15:
05:01:15:WU03:FS03:0x15:*------------------------------*
05:01:15:WU03:FS03:0x15:Folding@Home GPU Core
05:01:15:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
05:01:15:WU03:FS03:0x15:Build host             AmoebaRemote
05:01:15:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
05:01:15:WU03:FS03:0x15:Core                   15
05:01:15:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
05:01:15:WU03:FS03:0x15:
05:01:15:WU03:FS03:0x15:Window's signal control handler registered.
05:01:15:WU03:FS03:0x15:Preparing to commence simulation
05:01:15:WU03:FS03:0x15:- Looking at optimizations...
05:01:15:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
05:01:15:WU03:FS03:0x15:- Created dyn
05:01:15:WU03:FS03:0x15:- Files status OK
05:01:15:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
05:01:15:WU03:FS03:0x15:- Expanded 126784 -> 507182 (decompressed 400.0 percent)
05:01:15:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=126784 data_size=507182, decompressed_data_size=507182 diff=0
05:01:15:WU03:FS03:0x15:- Digital signature verified
05:01:15:WU03:FS03:0x15:
05:01:15:WU03:FS03:0x15:Project: 7626 (Run 11, Clone 0, Gen 148)
05:01:15:WU03:FS03:0x15:
05:01:15:WU03:FS03:0x15:Assembly optimizations on if available.
05:01:15:WU03:FS03:0x15:Entering M.D.
05:01:16:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  3545147320 523054169 171819085 3877915490 3484518196
05:01:16:WU03:FS03:0x15:GPU device id=1
05:01:16:WU03:FS03:0x15:Working on Protein
05:01:16:WU03:FS03:0x15:Client config unavailable.
05:01:16:WU03:FS03:0x15:Starting GUI Server
05:02:23:WU03:FS03:0x15:Setting checkpoint frequency: 400000
05:02:23:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
05:06:56:WU03:FS03:0x15:Completed    400000 out of 40000000 steps (1%).
05:11:29:WU03:FS03:0x15:Completed    800000 out of 40000000 steps (2%).
05:16:02:WU03:FS03:0x15:Completed   1200000 out of 40000000 steps (3%).
05:20:35:WU03:FS03:0x15:Completed   1600000 out of 40000000 steps (4%).
05:25:08:WU03:FS03:0x15:Completed   2000000 out of 40000000 steps (5%).
05:29:41:WU03:FS03:0x15:Completed   2400000 out of 40000000 steps (6%).
05:34:14:WU03:FS03:0x15:Completed   2800000 out of 40000000 steps (7%).
05:38:48:WU03:FS03:0x15:Completed   3200000 out of 40000000 steps (8%).
05:43:21:WU03:FS03:0x15:Completed   3600000 out of 40000000 steps (9%).
05:47:54:WU03:FS03:0x15:Completed   4000000 out of 40000000 steps (10%).
05:52:27:WU03:FS03:0x15:Completed   4400000 out of 40000000 steps (11%).
05:57:00:WU03:FS03:0x15:Completed   4800000 out of 40000000 steps (12%).
06:01:33:WU03:FS03:0x15:Completed   5200000 out of 40000000 steps (13%).
******************************** Date: 18/05/13 ********************************
06:06:06:WU03:FS03:0x15:Completed   5600000 out of 40000000 steps (14%).
06:09:57:WU03:FS03:0x15:Completed   6000000 out of 40000000 steps (15%).
06:09:57:WU03:FS03:0x15:mdrun_gpu returned 52
06:09:57:WU03:FS03:0x15:NANs detected on GPU
06:09:57:WU03:FS03:0x15:
06:09:57:WU03:FS03:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
06:09:57:WARNING:WU03:FS03:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
06:09:57:WU03:FS03:Starting
06:09:57:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
06:09:57:WU03:FS03:Started FahCore on PID 3764
06:09:57:WU03:FS03:Core PID:1336
06:09:57:WU03:FS03:FahCore 0x15 started
06:09:58:WU03:FS03:0x15:
06:09:58:WU03:FS03:0x15:*------------------------------*
06:09:58:WU03:FS03:0x15:Folding@Home GPU Core
06:09:58:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:09:58:WU03:FS03:0x15:Build host             AmoebaRemote
06:09:58:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
06:09:58:WU03:FS03:0x15:Core                   15
06:09:58:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
06:09:58:WU03:FS03:0x15:
06:09:58:WU03:FS03:0x15:Window's signal control handler registered.
06:09:58:WU03:FS03:0x15:Preparing to commence simulation
06:09:58:WU03:FS03:0x15:- Looking at optimizations...
06:09:58:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
06:09:59:WU03:FS03:0x15:- Created dyn
06:09:59:WU03:FS03:0x15:- Files status OK
06:09:59:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:09:59:WU03:FS03:0x15:- Expanded 126784 -> 507182 (decompressed 400.0 percent)
06:09:59:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=126784 data_size=507182, decompressed_data_size=507182 diff=0
06:09:59:WU03:FS03:0x15:- Digital signature verified
06:09:59:WU03:FS03:0x15:
06:09:59:WU03:FS03:0x15:Project: 7626 (Run 11, Clone 0, Gen 148)
06:09:59:WU03:FS03:0x15:
06:09:59:WU03:FS03:0x15:Assembly optimizations on if available.
06:09:59:WU03:FS03:0x15:Entering M.D.
06:10:01:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  3545147320 523054169 171819085 3877915490 3484518196
06:10:01:WU03:FS03:0x15:GPU device id=1
06:10:01:WU03:FS03:0x15:Working on Protein
06:10:01:WU03:FS03:0x15:Client config unavailable.
06:10:01:WU03:FS03:0x15:Starting GUI Server
06:11:07:WU03:FS03:0x15:Setting checkpoint frequency: 400000
06:11:07:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
06:11:08:WARNING:WU03:FS03:Detected clock skew (1 mins 09 secs), adjusting time estimates
06:15:40:WU03:FS03:0x15:Completed    400000 out of 40000000 steps (1%).
06:20:13:WU03:FS03:0x15:Completed    800000 out of 40000000 steps (2%).
06:24:46:WU03:FS03:0x15:Completed   1200000 out of 40000000 steps (3%).
06:29:19:WU03:FS03:0x15:Completed   1600000 out of 40000000 steps (4%).
06:33:54:WU03:FS03:0x15:Completed   2000000 out of 40000000 steps (5%).
06:38:27:WU03:FS03:0x15:Completed   2400000 out of 40000000 steps (6%).
06:43:00:WU03:FS03:0x15:Completed   2800000 out of 40000000 steps (7%).
06:47:32:WU03:FS03:0x15:Completed   3200000 out of 40000000 steps (8%).
06:52:05:WU03:FS03:0x15:Completed   3600000 out of 40000000 steps (9%).
06:56:38:WU03:FS03:0x15:Completed   4000000 out of 40000000 steps (10%).
07:01:12:WU03:FS03:0x15:Completed   4400000 out of 40000000 steps (11%).
07:05:44:WU03:FS03:0x15:Completed   4800000 out of 40000000 steps (12%).
07:10:17:WU03:FS03:0x15:Completed   5200000 out of 40000000 steps (13%).
07:14:50:WU03:FS03:0x15:Completed   5600000 out of 40000000 steps (14%).
07:19:23:WU03:FS03:0x15:Completed   6000000 out of 40000000 steps (15%).
07:23:56:WU03:FS03:0x15:Completed   6400000 out of 40000000 steps (16%).
07:28:29:WU03:FS03:0x15:Completed   6800000 out of 40000000 steps (17%).
07:33:02:WU03:FS03:0x15:Completed   7200000 out of 40000000 steps (18%).
07:37:35:WU03:FS03:0x15:Completed   7600000 out of 40000000 steps (19%).
07:42:08:WU03:FS03:0x15:Completed   8000000 out of 40000000 steps (20%).
07:46:05:WU03:FS03:0x15:Completed   8400000 out of 40000000 steps (21%).
07:46:05:WU03:FS03:0x15:mdrun_gpu returned 52
07:46:05:WU03:FS03:0x15:NANs detected on GPU
07:46:05:WU03:FS03:0x15:
07:46:05:WU03:FS03:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:46:05:WARNING:WU03:FS03:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:46:06:WU03:FS03:Starting
07:46:06:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
07:46:06:WU03:FS03:Started FahCore on PID 4992
07:46:06:WU03:FS03:Core PID:3984
07:46:06:WU03:FS03:FahCore 0x15 started
07:46:06:WU03:FS03:0x15:
07:46:06:WU03:FS03:0x15:*------------------------------*
07:46:06:WU03:FS03:0x15:Folding@Home GPU Core
07:46:06:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:46:06:WU03:FS03:0x15:Build host             AmoebaRemote
07:46:06:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
07:46:06:WU03:FS03:0x15:Core                   15
07:46:06:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
07:46:06:WU03:FS03:0x15:
07:46:06:WU03:FS03:0x15:Window's signal control handler registered.
07:46:06:WU03:FS03:0x15:Preparing to commence simulation
07:46:06:WU03:FS03:0x15:- Looking at optimizations...
07:46:06:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
07:46:07:WU03:FS03:0x15:- Created dyn
07:46:07:WU03:FS03:0x15:- Files status OK
07:46:07:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:46:07:WU03:FS03:0x15:- Expanded 126784 -> 507182 (decompressed 400.0 percent)
07:46:07:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=126784 data_size=507182, decompressed_data_size=507182 diff=0
07:46:07:WU03:FS03:0x15:- Digital signature verified
07:46:07:WU03:FS03:0x15:
07:46:07:WU03:FS03:0x15:Project: 7626 (Run 11, Clone 0, Gen 148)
07:46:07:WU03:FS03:0x15:
07:46:07:WU03:FS03:0x15:Assembly optimizations on if available.
07:46:07:WU03:FS03:0x15:Entering M.D.
07:46:09:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  3545147320 523054169 171819085 3877915490 3484518196
07:46:09:WU03:FS03:0x15:GPU device id=1
07:46:09:WU03:FS03:0x15:Working on Protein
07:46:09:WU03:FS03:0x15:Client config unavailable.
07:46:09:WU03:FS03:0x15:Starting GUI Server
07:47:15:WU03:FS03:0x15:Setting checkpoint frequency: 400000
07:47:15:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
07:47:16:WARNING:WU03:FS03:Detected clock skew (1 mins 09 secs), adjusting time estimates
07:51:48:WU03:FS03:0x15:Completed    400000 out of 40000000 steps (1%).
07:56:22:WU03:FS03:0x15:Completed    800000 out of 40000000 steps (2%).
08:00:55:WU03:FS03:0x15:Completed   1200000 out of 40000000 steps (3%).
08:05:28:WU03:FS03:0x15:Completed   1600000 out of 40000000 steps (4%).
08:10:01:WU03:FS03:0x15:Completed   2000000 out of 40000000 steps (5%).
08:14:34:WU03:FS03:0x15:Completed   2400000 out of 40000000 steps (6%).
08:19:07:WU03:FS03:0x15:Completed   2800000 out of 40000000 steps (7%).
08:23:40:WU03:FS03:0x15:Completed   3200000 out of 40000000 steps (8%).
08:28:14:WU03:FS03:0x15:Completed   3600000 out of 40000000 steps (9%).
08:32:47:WU03:FS03:0x15:Completed   4000000 out of 40000000 steps (10%).
08:37:20:WU03:FS03:0x15:Completed   4400000 out of 40000000 steps (11%).
08:41:53:WU03:FS03:0x15:Completed   4800000 out of 40000000 steps (12%).
08:46:26:WU03:FS03:0x15:Completed   5200000 out of 40000000 steps (13%).
08:50:59:WU03:FS03:0x15:Completed   5600000 out of 40000000 steps (14%).
08:55:33:WU03:FS03:0x15:Completed   6000000 out of 40000000 steps (15%).
09:00:06:WU03:FS03:0x15:Completed   6400000 out of 40000000 steps (16%).
09:04:40:WU03:FS03:0x15:Completed   6800000 out of 40000000 steps (17%).
09:09:13:WU03:FS03:0x15:Completed   7200000 out of 40000000 steps (18%).
09:13:46:WU03:FS03:0x15:Completed   7600000 out of 40000000 steps (19%).
09:18:19:WU03:FS03:0x15:Completed   8000000 out of 40000000 steps (20%).
09:22:52:WU03:FS03:0x15:Completed   8400000 out of 40000000 steps (21%).
09:27:26:WU03:FS03:0x15:Completed   8800000 out of 40000000 steps (22%).
09:31:59:WU03:FS03:0x15:Completed   9200000 out of 40000000 steps (23%).
09:36:32:WU03:FS03:0x15:Completed   9600000 out of 40000000 steps (24%).
09:41:05:WU03:FS03:0x15:Completed  10000000 out of 40000000 steps (25%).
09:45:39:WU03:FS03:0x15:Completed  10400000 out of 40000000 steps (26%).
09:50:12:WU03:FS03:0x15:Completed  10800000 out of 40000000 steps (27%).
09:54:45:WU03:FS03:0x15:Completed  11200000 out of 40000000 steps (28%).
09:59:18:WU03:FS03:0x15:Completed  11600000 out of 40000000 steps (29%).
10:03:25:WU03:FS03:0x15:Completed  12000000 out of 40000000 steps (30%).
10:03:25:WU03:FS03:0x15:mdrun_gpu returned 52
10:03:25:WU03:FS03:0x15:NANs detected on GPU
10:03:25:WU03:FS03:0x15:
10:03:25:WU03:FS03:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
10:03:25:WARNING:WU03:FS03:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
10:03:25:WU03:FS03:Starting
10:03:25:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
10:03:25:WU03:FS03:Started FahCore on PID 4672
10:03:25:WU03:FS03:Core PID:3340
10:03:25:WU03:FS03:FahCore 0x15 started
10:03:25:WU03:FS03:0x15:
10:03:25:WU03:FS03:0x15:*------------------------------*
10:03:25:WU03:FS03:0x15:Folding@Home GPU Core
10:03:25:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
10:03:25:WU03:FS03:0x15:Build host             AmoebaRemote
10:03:25:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
10:03:25:WU03:FS03:0x15:Core                   15
10:03:25:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
10:03:25:WU03:FS03:0x15:
10:03:25:WU03:FS03:0x15:Window's signal control handler registered.
10:03:25:WU03:FS03:0x15:Preparing to commence simulation
10:03:26:WU03:FS03:0x15:- Looking at optimizations...
10:03:26:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
10:03:27:WU03:FS03:0x15:- Created dyn
10:03:27:WU03:FS03:0x15:- Files status OK
10:03:27:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
10:03:27:WU03:FS03:0x15:- Expanded 126784 -> 507182 (decompressed 400.0 percent)
10:03:27:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=126784 data_size=507182, decompressed_data_size=507182 diff=0
10:03:27:WU03:FS03:0x15:- Digital signature verified
10:03:27:WU03:FS03:0x15:
10:03:27:WU03:FS03:0x15:Project: 7626 (Run 11, Clone 0, Gen 148)
10:03:27:WU03:FS03:0x15:
10:03:27:WU03:FS03:0x15:Assembly optimizations on if available.
10:03:27:WU03:FS03:0x15:Entering M.D.
10:03:29:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  3545147320 523054169 171819085 3877915490 3484518196
10:03:29:WU03:FS03:0x15:GPU device id=1
10:03:29:WU03:FS03:0x15:Working on Protein
10:03:29:WU03:FS03:0x15:Client config unavailable.
10:03:29:WU03:FS03:0x15:Starting GUI Server
10:04:35:WU03:FS03:0x15:Setting checkpoint frequency: 400000
10:04:35:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
10:04:36:WARNING:WU03:FS03:Detected clock skew (1 mins 10 secs), adjusting time estimates
10:09:08:WU03:FS03:0x15:Completed    400000 out of 40000000 steps (1%).
10:13:42:WU03:FS03:0x15:Completed    800000 out of 40000000 steps (2%).
10:18:15:WU03:FS03:0x15:Completed   1200000 out of 40000000 steps (3%).
10:22:48:WU03:FS03:0x15:Completed   1600000 out of 40000000 steps (4%).
10:27:21:WU03:FS03:0x15:Completed   2000000 out of 40000000 steps (5%).
10:31:54:WU03:FS03:0x15:Completed   2400000 out of 40000000 steps (6%).
10:36:27:WU03:FS03:0x15:Completed   2800000 out of 40000000 steps (7%).
10:41:01:WU03:FS03:0x15:Completed   3200000 out of 40000000 steps (8%).
10:45:34:WU03:FS03:0x15:Completed   3600000 out of 40000000 steps (9%).
10:50:07:WU03:FS03:0x15:Completed   4000000 out of 40000000 steps (10%).
10:54:40:WU03:FS03:0x15:Completed   4400000 out of 40000000 steps (11%).
10:59:13:WU03:FS03:0x15:Completed   4800000 out of 40000000 steps (12%).
11:03:46:WU03:FS03:0x15:Completed   5200000 out of 40000000 steps (13%).
11:08:20:WU03:FS03:0x15:Completed   5600000 out of 40000000 steps (14%).
11:12:53:WU03:FS03:0x15:Completed   6000000 out of 40000000 steps (15%).
11:17:26:WU03:FS03:0x15:Completed   6400000 out of 40000000 steps (16%).
11:21:59:WU03:FS03:0x15:Completed   6800000 out of 40000000 steps (17%).
11:26:32:WU03:FS03:0x15:Completed   7200000 out of 40000000 steps (18%).
11:31:05:WU03:FS03:0x15:Completed   7600000 out of 40000000 steps (19%).
11:35:39:WU03:FS03:0x15:Completed   8000000 out of 40000000 steps (20%).
11:40:12:WU03:FS03:0x15:Completed   8400000 out of 40000000 steps (21%).
11:44:45:WU03:FS03:0x15:Completed   8800000 out of 40000000 steps (22%).
11:49:18:WU03:FS03:0x15:Completed   9200000 out of 40000000 steps (23%).
11:53:51:WU03:FS03:0x15:Completed   9600000 out of 40000000 steps (24%).
11:58:24:WU03:FS03:0x15:Completed  10000000 out of 40000000 steps (25%).
12:02:57:WU03:FS03:0x15:Completed  10400000 out of 40000000 steps (26%).
******************************** Date: 18/05/13 ********************************
12:07:30:WU03:FS03:0x15:Completed  10800000 out of 40000000 steps (27%).
12:12:03:WU03:FS03:0x15:Completed  11200000 out of 40000000 steps (28%).
12:16:37:WU03:FS03:0x15:Completed  11600000 out of 40000000 steps (29%).
12:21:10:WU03:FS03:0x15:Completed  12000000 out of 40000000 steps (30%).
12:25:43:WU03:FS03:0x15:Completed  12400000 out of 40000000 steps (31%).
12:29:58:WU03:FS03:0x15:Completed  12800000 out of 40000000 steps (32%).
12:29:58:WU03:FS03:0x15:mdrun_gpu returned 52
12:29:58:WU03:FS03:0x15:NANs detected on GPU
12:29:58:WU03:FS03:0x15:
12:29:58:WU03:FS03:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
12:29:59:WARNING:WU03:FS03:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
12:29:59:WU03:FS03:Starting
12:29:59:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
12:29:59:WU03:FS03:Started FahCore on PID 4944
12:29:59:WU03:FS03:Core PID:2060
12:29:59:WU03:FS03:FahCore 0x15 started
12:29:59:WU03:FS03:0x15:
12:29:59:WU03:FS03:0x15:*------------------------------*
12:29:59:WU03:FS03:0x15:Folding@Home GPU Core
12:29:59:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
12:29:59:WU03:FS03:0x15:Build host             AmoebaRemote
12:29:59:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
12:29:59:WU03:FS03:0x15:Core                   15
12:29:59:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
12:29:59:WU03:FS03:0x15:
12:29:59:WU03:FS03:0x15:Window's signal control handler registered.
12:29:59:WU03:FS03:0x15:Preparing to commence simulation
12:29:59:WU03:FS03:0x15:- Looking at optimizations...
12:29:59:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
12:30:00:WU03:FS03:0x15:- Created dyn
12:30:00:WU03:FS03:0x15:- Files status OK
12:30:00:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
12:30:00:WU03:FS03:0x15:- Expanded 126784 -> 507182 (decompressed 400.0 percent)
12:30:00:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=126784 data_size=507182, decompressed_data_size=507182 diff=0
12:30:00:WU03:FS03:0x15:- Digital signature verified
12:30:00:WU03:FS03:0x15:
12:30:00:WU03:FS03:0x15:Project: 7626 (Run 11, Clone 0, Gen 148)
12:30:00:WU03:FS03:0x15:
12:30:00:WU03:FS03:0x15:Assembly optimizations on if available.
12:30:00:WU03:FS03:0x15:Entering M.D.
12:30:02:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  3545147320 523054169 171819085 3877915490 3484518196
12:30:02:WU03:FS03:0x15:GPU device id=1
12:30:02:WU03:FS03:0x15:Working on Protein
12:30:02:WU03:FS03:0x15:Client config unavailable.
12:30:02:WU03:FS03:0x15:Starting GUI Server
12:31:09:WU03:FS03:0x15:Setting checkpoint frequency: 400000
12:31:09:WARNING:WU03:FS03:Detected clock skew (1 mins 09 secs), adjusting time estimates
12:31:09:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
12:33:30:WU03:FS03:0x15:Client no longer detected. Shutting down core 
12:33:30:WU03:FS03:0x15:
12:33:30:WU03:FS03:0x15:Folding@home Core Shutdown: CLIENT_DIED
12:33:31:WU03:FS03:FahCore returned: INTERRUPTED (102 = 0x66)
12:33:47:WU03:FS03:Starting
12:33:47:WU03:FS03:Running FahCore: "E:\V7 folding\FAHClient/FAHCoreWrapper.exe" "E:/V7 folding/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 03 -suffix 01 -version 702 -lifeline 3800 -checkpoint 15 -gpu 1
12:33:47:WU03:FS03:Started FahCore on PID 3360
12:33:47:WU03:FS03:Core PID:3264
12:33:47:WU03:FS03:FahCore 0x15 started
12:33:47:WARNING:WU03:FS03:FahCore returned: MISSING_WORK_FILES (116 = 0x74)
12:33:47:WARNING:WU03:FS03:Fatal error, dumping
12:33:47:WU03:FS03:Sending unit results: id:03 state:SEND error:DUMPED project:7626 run:11 clone:0 gen:148 core:0x15 unit:0x000000dd664f2dd14fe619ce9b068f27
12:33:47:WARNING:WU03:FS03:Missing original Unit data, cannot send dump report
12:33:48:WU03:FS03:Cleaning up
Top
bollix47
Posts: 2963
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: 7626 run:11 clone:0 gen:148

Post by bollix47 »

Thank you for your report.

There are no returns for this WU in the database at this time.

I have marked it for follow-up.

Since the failures occurred at different percentage points you might want to check your temperatures and power consumption.
Top
rickoic
Posts: 320
Joined: Sat May 23, 2009 4:49 pm
Hardware configuration: eVga x299 DARK 2070 Super, eVGA 2080, eVga 1070, eVga 2080 Super
MSI x399 eVga 2080, eVga 1070, eVga 1070, GT970
Location: Mississippi near Memphis, Tn

Re: 7626 run:11 clone:0 gen:148

Post by rickoic »

Noticed that each failure ended with NAN's being reported on your GPU.

Tks
Rick
I'm folding because Dec 2005 I had radical prostate surgery.
Lost brother to spinal cancer, brother-in-law to prostate cancer.
Several 1st cousins lost and a few who have survived.
Top
wrinvert
Posts: 16
Joined: Sat Oct 22, 2011 8:24 am

Re: 7626 run:11 clone:0 gen:148

Post by wrinvert »

temps are not an issue im on water and never above 50'C on a 680 and its all backed by a nex 1500 psu running at 50%. the whole set up has been running for days without issue on 7663 and has always ran the 762X without issue before. this is the first one that ive seen ever error on me and the first one to give the NaNs issue.

the there a explanation of what NaNs issue is somewhere?
Top
bollix47
Posts: 2963
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: 7626 run:11 clone:0 gen:148

Post by bollix47 »

Thank you for the additional information ... we will continue to monitor the database and for any other reports of failure for this work unit.

Status 7a
Top
rickoic
Posts: 320
Joined: Sat May 23, 2009 4:49 pm
Hardware configuration: eVga x299 DARK 2070 Super, eVGA 2080, eVga 1070, eVga 2080 Super
MSI x399 eVga 2080, eVga 1070, eVga 1070, GT970
Location: Mississippi near Memphis, Tn

Re: 7626 run:11 clone:0 gen:148

Post by rickoic »

NAN is short for Not A Number which means that for some reason the result returned by the math computations wasn't interpeted as a number but the hexidecimal equilivent of a letter or special character.

Like on your keyboard when you press a key 1 of 256 combinations of hexidecimal is sent to your computer to interept and send to where it's susposed to go. A lower case a = Hex 61, Capital A = Hex 41. Numbers are Hex 30, 31, 32, 33, 34, 35, 36, 37, 38, and 39.
If you google ASCII Table you can see what all the other combinations are.

Rick
I'm folding because Dec 2005 I had radical prostate surgery.
Lost brother to spinal cancer, brother-in-law to prostate cancer.
Several 1st cousins lost and a few who have survived.
Top
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:
Contact 7im

Re: 7626 run:11 clone:0 gen:148

Post by 7im »

NaN errors are computational errors. On the CPU client almost always meant hardware problems, too much overclock, memory timings set too aggressively, stuff like that, although rarely a bad WU could cause one also. I have to assume GPUs are the same way for NaN errors.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Top
bollix47
Posts: 2963
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: 7626 run:11 clone:0 gen:148

Post by bollix47 »

Another folder was able to complete this work unit:

Hi xxxx(team xxxx),
Your WU (P7626 R11 C0 G148) was added to the stats database on 2013-05-26 08:10:06 for 14093 points of credit.

Report closed.
Top
Ricky
Posts: 474
Joined: Sat Aug 01, 2015 1:34 am
Hardware configuration: 1. 2 each E5-2630 V3 processors, 64 GB RAM, GTX980SC GPU, and GTX980 GPU running on windows 8.1 operating system.
2. I7-6950X V3 processor, 32 GB RAM, 1 GTX980tiFTW, and 2 each GTX1080FTW GPUs running on windows 8.1 operating system.
Location: New Mexico

Re: 7626 run:11 clone:0 gen:148

Post by Ricky »

I have noticed that Project 7626 returns 14093 points no mater how fast it is completed.
Top
Nathan_P
Posts: 1164
Joined: Wed Apr 01, 2009 9:22 pm
Hardware configuration: Asus Z8NA D6C, 2 [email protected] Ghz, , 12gb Ram, GTX 980ti, AX650 PSU, win 10 (daily use)

Asus Z87 WS, Xeon E3-1230L v3, 8gb ram, KFA GTX 1080, EVGA 750ti , AX760 PSU, Mint 18.2 OS

Not currently folding
Asus Z9PE- D8 WS, 2 [email protected] Ghz, 16Gb 1.35v Ram, Ubuntu (Fold only)
Asus Z9PA, 2 Ivy 12 core, 16gb Ram, H folding appliance (fold only)
Location: Jersey, Channel islands

Re: 7626 run:11 clone:0 gen:148

Post by Nathan_P »

Ricky wrote:I have noticed that Project 7626 returns 14093 points no mater how fast it is completed.
The Quick return bonus does not apply to any core 15 WU.
Image
Top
Post Reply

Return to “Issues with a specific WU”