Folding Forum

Posted: **Sun Sep 19, 2010 10:55 am**

Hi!
about 30 minutes ago, my gpu client exceded EUE limit in just a little over 2 minutes when encountered the wu P5771 (Run 10, Clone 157, Gen 851).
From what I remember, this is not the first time I fold a P5771 wu, so i don't think it's a whole project issue.
here is part of FAHlog, when the problem occurred:

Code: Select all

[10:20:14] 
[10:20:14] + Processing work unit
[10:20:14] Core required: FahCore_11.exe
[10:20:14] Core found.
[10:20:14] Working on queue slot 08 [September 19 10:20:14 UTC]
[10:20:14] + Working ...
[10:20:14] - Calling '.\FahCore_11.exe -dir work/ -suffix 08 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 14260 -version 623'

[10:20:15] 
[10:20:15] *------------------------------*
[10:20:15] Folding@Home GPU Core
[10:20:15] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[10:20:15] 
[10:20:15] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[10:20:15] Build host: amoeba
[10:20:15] Board Type: Nvidia
[10:20:15] Core      : 
[10:20:15] Preparing to commence simulation
[10:20:15] - Assembly optimizations manually forced on.
[10:20:15] - Not checking prior termination.
[10:20:15] - Expanded 45433 -> 251112 (decompressed 552.7 percent)
[10:20:15] Called DecompressByteArray: compressed_data_size=45433 data_size=251112, decompressed_data_size=251112 diff=0
[10:20:15] - Digital signature verified
[10:20:15] 
[10:20:15] Project: 5771 (Run 10, Clone 157, Gen 851)
[10:20:15] 
[10:20:15] Assembly optimizations on if available.
[10:20:15] Entering M.D.
[10:20:21] Tpr hash work/wudata_08.tpr:  4254969175 3030929658 2442498514 2645022649 1179656556
[10:20:21] 
[10:20:21] Calling fah_main args: 14 usage=100
[10:20:21] 
[10:20:21] mdrun_gpu returned 
[10:20:21] Going to send back what have done -- stepsTotalG=0
[10:20:21] Work fraction=0.0000 steps=0.
[10:20:25] logfile size=4949 infoLength=4949 edr=0 trr=25
[10:20:25] + Opened results file
[10:20:25] - Writing 5487 bytes of core data to disk...
[10:20:25] Done: 4975 -> 1858 (compressed to 37.3 percent)
[10:20:25]   ... Done.
[10:20:25] DeleteFrameFiles: successfully deleted file=work/wudata_08.ckp
[10:20:25] 
[10:20:25] Folding@home Core Shutdown: UNSTABLE_MACHINE
[10:20:29] CoreStatus = 7A (122)
[10:20:29] Sending work to server
[10:20:29] Project: 5771 (Run 10, Clone 157, Gen 851)


[10:20:29] + Attempting to send results [September 19 10:20:29 UTC]
[10:20:29] - Reading file work/wuresults_08.dat from core
[10:20:29]   (Read 2370 bytes from disk)
[10:20:29] Connecting to http://171.67.108.11:8080/
[10:20:30] Posted data.
[10:20:30] Initial: 0000; - Uploaded at ~3 kB/s
[10:20:30] - Averaged speed for that direction ~38 kB/s
[10:20:30] + Results successfully sent
[10:20:30] Thank you for your contribution to Folding@Home.
[10:20:34] Trying to send all finished work units
[10:20:34] + No unsent completed units remaining.
[10:20:34] - Preparing to get new work unit...
[10:20:34] + Attempting to get work packet
[10:20:34] - Will indicate memory of 4094 MB
[10:20:34] - Connecting to assignment server
[10:20:34] Connecting to http://assign-GPU.stanford.edu:8080/
[10:20:35] Posted data.
[10:20:35] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[10:20:35] + News From Folding@Home: Welcome to Folding@Home
[10:20:35] Loaded queue successfully.
[10:20:35] Connecting to http://171.67.108.11:8080/
[10:20:36] Posted data.
[10:20:36] Initial: 0000; - Receiving payload (expected size: 45853)
[10:20:37] - Downloaded at ~44 kB/s
[10:20:37] - Averaged speed for that direction ~65 kB/s
[10:20:37] + Received work.
[10:20:37] Trying to send all finished work units
[10:20:37] + No unsent completed units remaining.
[10:20:37] + Closed connections
[10:20:42] 
[10:20:42] + Processing work unit
[10:20:42] Core required: FahCore_11.exe
[10:20:42] Core found.
[10:20:42] Working on queue slot 09 [September 19 10:20:42 UTC]
[10:20:42] + Working ...
[10:20:42] - Calling '.\FahCore_11.exe -dir work/ -suffix 09 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 14260 -version 623'

[10:20:42] 
[10:20:42] *------------------------------*
[10:20:42] Folding@Home GPU Core
[10:20:42] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[10:20:42] 
[10:20:42] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[10:20:42] Build host: amoeba
[10:20:42] Board Type: Nvidia
[10:20:42] Core      : 
[10:20:42] Preparing to commence simulation
[10:20:42] - Assembly optimizations manually forced on.
[10:20:42] - Not checking prior termination.
[10:20:42] - Expanded 45341 -> 251112 (decompressed 553.8 percent)
[10:20:42] Called DecompressByteArray: compressed_data_size=45341 data_size=251112, decompressed_data_size=251112 diff=0
[10:20:42] - Digital signature verified
[10:20:42] 
[10:20:42] Project: 5772 (Run 7, Clone 273, Gen 2457)
[10:20:42] 
[10:20:42] Assembly optimizations on if available.
[10:20:42] Entering M.D.
[10:20:48] Tpr hash work/wudata_09.tpr:  3770870207 2497586244 1287383949 1152755512 709232897
[10:20:48] 
[10:20:48] Calling fah_main args: 14 usage=100
[10:20:48] 
[10:20:48] mdrun_gpu returned 
[10:20:48] Going to send back what have done -- stepsTotalG=0
[10:20:48] Work fraction=0.0000 steps=0.
[10:20:52] logfile size=4949 infoLength=4949 edr=0 trr=25
[10:20:52] + Opened results file
[10:20:52] - Writing 5487 bytes of core data to disk...
[10:20:52] Done: 4975 -> 1847 (compressed to 37.1 percent)
[10:20:52]   ... Done.
[10:20:52] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[10:20:52] 
[10:20:52] Folding@home Core Shutdown: UNSTABLE_MACHINE
[10:20:56] CoreStatus = 7A (122)
[10:20:56] Sending work to server
[10:20:56] Project: 5772 (Run 7, Clone 273, Gen 2457)


[10:20:56] + Attempting to send results [September 19 10:20:56 UTC]
[10:20:56] - Reading file work/wuresults_09.dat from core
[10:20:56]   (Read 2359 bytes from disk)
[10:20:56] Connecting to http://171.67.108.11:8080/
[10:20:57] Posted data.
[10:20:57] Initial: 0000; - Uploaded at ~3 kB/s
[10:20:57] - Averaged speed for that direction ~31 kB/s
[10:20:57] + Results successfully sent
[10:20:57] Thank you for your contribution to Folding@Home.
[10:21:01] Trying to send all finished work units
[10:21:01] + No unsent completed units remaining.
[10:21:01] - Preparing to get new work unit...
[10:21:01] + Attempting to get work packet
[10:21:01] - Will indicate memory of 4094 MB
[10:21:01] - Connecting to assignment server
[10:21:01] Connecting to http://assign-GPU.stanford.edu:8080/
[10:21:03] Posted data.
[10:21:03] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[10:21:03] + News From Folding@Home: Welcome to Folding@Home
[10:21:03] Loaded queue successfully.
[10:21:03] Connecting to http://171.67.108.11:8080/
[10:21:04] Posted data.
[10:21:04] Initial: 0000; - Receiving payload (expected size: 45834)
[10:21:04] Conversation time very short, giving reduced weight in bandwidth avg
[10:21:04] - Downloaded at ~89 kB/s
[10:21:04] - Averaged speed for that direction ~68 kB/s
[10:21:04] + Received work.
[10:21:04] Trying to send all finished work units
[10:21:04] + No unsent completed units remaining.
[10:21:04] + Closed connections
[10:21:09] 
[10:21:09] + Processing work unit
[10:21:09] Core required: FahCore_11.exe
[10:21:09] Core found.
[10:21:09] Working on queue slot 00 [September 19 10:21:09 UTC]
[10:21:09] + Working ...
[10:21:09] - Calling '.\FahCore_11.exe -dir work/ -suffix 00 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 14260 -version 623'

[10:21:09] 
[10:21:09] *------------------------------*
[10:21:09] Folding@Home GPU Core
[10:21:09] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[10:21:09] 
[10:21:09] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[10:21:09] Build host: amoeba
[10:21:09] Board Type: Nvidia
[10:21:09] Core      : 
[10:21:09] Preparing to commence simulation
[10:21:09] - Assembly optimizations manually forced on.
[10:21:09] - Not checking prior termination.
[10:21:09] - Expanded 45322 -> 251112 (decompressed 554.0 percent)
[10:21:09] Called DecompressByteArray: compressed_data_size=45322 data_size=251112, decompressed_data_size=251112 diff=0
[10:21:09] - Digital signature verified
[10:21:09] 
[10:21:09] Project: 5770 (Run 14, Clone 51, Gen 950)
[10:21:09] 
[10:21:09] Assembly optimizations on if available.
[10:21:09] Entering M.D.
[10:21:16] Tpr hash work/wudata_00.tpr:  3156104227 1605396293 2443059991 2260504931 4200505539
[10:21:16] 
[10:21:16] Calling fah_main args: 14 usage=100
[10:21:16] 
[10:21:16] mdrun_gpu returned 
[10:21:16] Going to send back what have done -- stepsTotalG=0
[10:21:16] Work fraction=0.0000 steps=0.
[10:21:20] logfile size=4947 infoLength=4947 edr=0 trr=25
[10:21:20] + Opened results file
[10:21:20] - Writing 5485 bytes of core data to disk...
[10:21:20] Done: 4973 -> 1860 (compressed to 37.4 percent)
[10:21:20]   ... Done.
[10:21:20] DeleteFrameFiles: successfully deleted file=work/wudata_00.ckp
[10:21:20] 
[10:21:20] Folding@home Core Shutdown: UNSTABLE_MACHINE
[10:21:24] CoreStatus = 7A (122)
[10:21:24] Sending work to server
[10:21:24] Project: 5770 (Run 14, Clone 51, Gen 950)


[10:21:24] + Attempting to send results [September 19 10:21:24 UTC]
[10:21:24] - Reading file work/wuresults_00.dat from core
[10:21:24]   (Read 2372 bytes from disk)
[10:21:24] Connecting to http://171.67.108.11:8080/
[10:21:25] Posted data.
[10:21:25] Initial: 0000; - Uploaded at ~3 kB/s
[10:21:25] - Averaged speed for that direction ~25 kB/s
[10:21:25] + Results successfully sent
[10:21:25] Thank you for your contribution to Folding@Home.
[10:21:29] Trying to send all finished work units
[10:21:29] + No unsent completed units remaining.
[10:21:29] - Preparing to get new work unit...
[10:21:29] + Attempting to get work packet
[10:21:29] - Will indicate memory of 4094 MB
[10:21:29] - Connecting to assignment server
[10:21:29] Connecting to http://assign-GPU.stanford.edu:8080/
[10:21:30] Posted data.
[10:21:30] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[10:21:30] + News From Folding@Home: Welcome to Folding@Home
[10:21:30] Loaded queue successfully.
[10:21:30] Connecting to http://171.67.108.11:8080/
[10:21:31] Posted data.
[10:21:31] Initial: 0000; - Receiving payload (expected size: 47191)
[10:21:32] - Downloaded at ~46 kB/s
[10:21:32] - Averaged speed for that direction ~64 kB/s
[10:21:32] + Received work.
[10:21:32] Trying to send all finished work units
[10:21:32] + No unsent completed units remaining.
[10:21:32] + Closed connections
[10:21:37] 
[10:21:37] + Processing work unit
[10:21:37] Core required: FahCore_11.exe
[10:21:37] Core found.
[10:21:37] Working on queue slot 01 [September 19 10:21:37 UTC]
[10:21:37] + Working ...
[10:21:37] - Calling '.\FahCore_11.exe -dir work/ -suffix 01 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 14260 -version 623'

[10:21:37] 
[10:21:37] *------------------------------*
[10:21:37] Folding@Home GPU Core
[10:21:37] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[10:21:37] 
[10:21:37] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[10:21:37] Build host: amoeba
[10:21:37] Board Type: Nvidia
[10:21:37] Core      : 
[10:21:37] Preparing to commence simulation
[10:21:37] - Assembly optimizations manually forced on.
[10:21:37] - Not checking prior termination.
[10:21:37] - Expanded 46679 -> 252912 (decompressed 541.8 percent)
[10:21:37] Called DecompressByteArray: compressed_data_size=46679 data_size=252912, decompressed_data_size=252912 diff=0
[10:21:37] - Digital signature verified
[10:21:37] 
[10:21:37] Project: 5766 (Run 13, Clone 189, Gen 966)
[10:21:37] 
[10:21:37] Assembly optimizations on if available.
[10:21:37] Entering M.D.
[10:21:43] Tpr hash work/wudata_01.tpr:  2730431744 1234762441 1336580278 3578391354 4084144945
[10:21:43] 
[10:21:43] Calling fah_main args: 14 usage=100
[10:21:43] 
[10:21:43] mdrun_gpu returned 
[10:21:43] Going to send back what have done -- stepsTotalG=0
[10:21:43] Work fraction=0.0000 steps=0.
[10:21:47] logfile size=4949 infoLength=4949 edr=0 trr=25
[10:21:47] + Opened results file
[10:21:47] - Writing 5487 bytes of core data to disk...
[10:21:47] Done: 4975 -> 1855 (compressed to 37.2 percent)
[10:21:47]   ... Done.
[10:21:47] DeleteFrameFiles: successfully deleted file=work/wudata_01.ckp
[10:21:47] 
[10:21:47] Folding@home Core Shutdown: UNSTABLE_MACHINE
[10:21:52] CoreStatus = 7A (122)
[10:21:52] Sending work to server
[10:21:52] Project: 5766 (Run 13, Clone 189, Gen 966)


[10:21:52] + Attempting to send results [September 19 10:21:52 UTC]
[10:21:52] - Reading file work/wuresults_01.dat from core
[10:21:52]   (Read 2367 bytes from disk)
[10:21:52] Connecting to http://171.67.108.11:8080/
[10:21:52] Posted data.
[10:21:52] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[10:21:52] - Uploaded at ~6 kB/s
[10:21:52] - Averaged speed for that direction ~23 kB/s
[10:21:52] + Results successfully sent
[10:21:52] Thank you for your contribution to Folding@Home.
[10:21:56] Trying to send all finished work units
[10:21:56] + No unsent completed units remaining.
[10:21:56] - Preparing to get new work unit...
[10:21:56] + Attempting to get work packet
[10:21:56] - Will indicate memory of 4094 MB
[10:21:56] - Connecting to assignment server
[10:21:56] Connecting to http://assign-GPU.stanford.edu:8080/
[10:21:58] Posted data.
[10:21:58] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[10:21:58] + News From Folding@Home: Welcome to Folding@Home
[10:21:58] Loaded queue successfully.
[10:21:58] Connecting to http://171.67.108.11:8080/
[10:21:59] Posted data.
[10:21:59] Initial: 0000; - Receiving payload (expected size: 47189)
[10:22:00] - Downloaded at ~46 kB/s
[10:22:00] - Averaged speed for that direction ~60 kB/s
[10:22:00] + Received work.
[10:22:00] Trying to send all finished work units
[10:22:00] + No unsent completed units remaining.
[10:22:00] + Closed connections
[10:22:05] 
[10:22:05] + Processing work unit
[10:22:05] Core required: FahCore_11.exe
[10:22:05] Core found.
[10:22:05] Working on queue slot 02 [September 19 10:22:05 UTC]
[10:22:05] + Working ...
[10:22:05] - Calling '.\FahCore_11.exe -dir work/ -suffix 02 -nocpulock -checkpoint 3 -forceasm -verbose -lifeline 14260 -version 623'

[10:22:05] 
[10:22:05] *------------------------------*
[10:22:05] Folding@Home GPU Core
[10:22:05] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[10:22:05] 
[10:22:05] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[10:22:05] Build host: amoeba
[10:22:05] Board Type: Nvidia
[10:22:05] Core      : 
[10:22:05] Preparing to commence simulation
[10:22:05] - Assembly optimizations manually forced on.
[10:22:05] - Not checking prior termination.
[10:22:05] - Expanded 46677 -> 252912 (decompressed 541.8 percent)
[10:22:05] Called DecompressByteArray: compressed_data_size=46677 data_size=252912, decompressed_data_size=252912 diff=0
[10:22:05] - Digital signature verified
[10:22:05] 
[10:22:05] Project: 5766 (Run 12, Clone 313, Gen 842)
[10:22:05] 
[10:22:05] Assembly optimizations on if available.
[10:22:05] Entering M.D.
[10:22:11] Tpr hash work/wudata_02.tpr:  2071452496 1300566072 4182949195 2568257064 2151227968
[10:22:11] 
[10:22:11] Calling fah_main args: 14 usage=100
[10:22:11] 
[10:22:11] mdrun_gpu returned 
[10:22:11] Going to send back what have done -- stepsTotalG=0
[10:22:11] Work fraction=0.0000 steps=0.
[10:22:15] logfile size=4948 infoLength=4948 edr=0 trr=25
[10:22:15] + Opened results file
[10:22:15] - Writing 5486 bytes of core data to disk...
[10:22:15] Done: 4974 -> 1854 (compressed to 37.2 percent)
[10:22:15]   ... Done.
[10:22:15] DeleteFrameFiles: successfully deleted file=work/wudata_02.ckp
[10:22:15] 
[10:22:15] Folding@home Core Shutdown: UNSTABLE_MACHINE
[10:22:19] CoreStatus = 7A (122)
[10:22:19] Sending work to server
[10:22:19] Project: 5766 (Run 12, Clone 313, Gen 842)


[10:22:19] + Attempting to send results [September 19 10:22:19 UTC]
[10:22:19] - Reading file work/wuresults_02.dat from core
[10:22:19]   (Read 2366 bytes from disk)
[10:22:19] Connecting to http://171.67.108.11:8080/
[10:22:20] Posted data.
[10:22:20] Initial: 0000; - Uploaded at ~3 kB/s
[10:22:20] - Averaged speed for that direction ~19 kB/s
[10:22:20] + Results successfully sent
[10:22:20] Thank you for your contribution to Folding@Home.
[10:22:24] EUE limit exceeded. Pausing 24 hours.

Posted: **Sun Sep 19, 2010 11:03 am**

Welcome to the F@H Forum edalecu,

From your FAHlog, I have detected the following WUs that returned "UNSTABLE_MACHINE"
[10:20:29] Project: 5771 (Run 10, Clone 157, Gen 851) <- You reported this one
[10:20:56] Project: 5772 (Run 7, Clone 273, Gen 2457)
[10:21:24] Project: 5770 (Run 14, Clone 51, Gen 950)
[10:21:52] Project: 5766 (Run 13, Clone 189, Gen 966)
[10:22:19] Project: 5766 (Run 12, Clone 313, Gen 842)

I am guessing that you have an unstable Overclock or a faulty GPU. Please give us more details so we can help you.

Posted: **Sun Sep 19, 2010 12:04 pm**

thanks for the reply!
My gpu is a gf 9600GSO 768MB GDDR3. I overclocked it from 600 to 630 MHz the gpu, 800 to 945 MHz the memory and 1500 to 1566 MHz the shaders.
I ran many tests some time ago, and it prooved me it's stable at folding and in games or even stress tests like furmark or physics fluidmark.
How do you recommend to test my overclock even more, so i'll be sure it's really stable? (supposing the overclock is the reason of failure on those wu-s)

Posted: **Sun Sep 19, 2010 12:38 pm**

Do you try MemtestG80 and/or MemtestCL ?

Posted: **Sun Sep 19, 2010 1:42 pm**

not yet. I'll try those tonight.
10x!

Posted: **Mon Sep 20, 2010 8:07 am**

Tested the graphics card with memtestG80 and memtestCL and there is no problem at all.
I don't know what could have been the issue there, though.

Posted: **Mon Sep 20, 2010 4:29 pm**

FWIW, another folder was able to complete this WU.

(P5771 R10 C157 G851) was added to the stats database on 2010-09-19 14:09:25 for 353 points of credit.

Posted: **Mon Sep 20, 2010 8:50 pm**

It definately have a problem because, during today, it encountered another 10 wus that gave UNSTABLE_MACHINE.
I lowered the memory speed from 945 to 928 MHz and will see if it's solid or not.
@sortofageek: how did you find out that that specific wu was completed?..or it was only an assumption?

Posted: **Mon Sep 20, 2010 8:57 pm**

Given the exact work unit info (project number, run, clone and gen), such as Project: 5771 (Run 10, Clone 157, Gen 851) which identifies the exact work unit, moderators can look up the work unit in a mod database to see if anyone has returned it, plus a little bit of info about the return. We can tell who returned it, whether it was a partial return or completely successful and how much credit was awarded.

Posted: **Wed Sep 22, 2010 1:25 am**

thanks. yesterday i got the 3rd eue on gpu client. I really don't know what is the reason,being known that I didn't get any problems with my gfx card in the last few weeks, from the moment I overclocked and tested it. #-o

I would like to know what component impacts most the folding performance: gpu, memory or shader frequency?... if anyone tested and compared results in different overclocking combinations with the same card and drew some conclusions about that, please tell me.

edit: I was thinking: could it be a driver problem?.. or not?

Posted: **Wed Sep 22, 2010 4:49 am**

OCing the Shaders yields the highest change in the PPD. Memory and Core frequencies not that much so are usually left at default by 24/7 folders.

Folding Forum

Project: 5771 (Run 10, Clone 157, Gen 851)

Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)

Re: Project: 5771 (Run 10, Clone 157, Gen 851)