When assigning new CPU WU, GPU WU is interrupted

Moderators: Site Moderators, FAHC Science Team

Post Reply
Nuon
Posts: 3
Joined: Sun Nov 24, 2024 1:03 pm

When assigning new CPU WU, GPU WU is interrupted

Post by Nuon »

With Version 8.4.9 the GPU WU is interrupted when a new CPU WU assignment is requested:

Code: Select all

23:10:31:I1:WU26:Completed 3350000 out of 5000000 steps (67%)
23:11:58:I1:WU25:Completed 2475000 out of 2500000 steps (99%)
23:12:53:I1:WU26:Completed 3400000 out of 5000000 steps (68%)
23:13:36:I1:WU25:Completed 2500000 out of 2500000 steps (100%)
23:13:36:I1:WU25:Average performance: 44.0816 ns/day
23:13:37:I1:WU25:Checkpoint completed at step 2500000
23:13:47:I1:WU25:Saving result file ..\logfile_01.txt
23:13:47:I1:WU25:Saving result file checkpointIntegrator.xml
23:13:47:I1:WU25:Saving result file checkpointState.xml
23:13:52:I1:WU25:Saving result file positions.xtc
23:13:53:I1:WU25:Saving result file science.log
23:13:53:I1:WU25:Saving result file xtcAtoms.csv.bz2
23:13:53:I1:WU25:Folding@home Core Shutdown: FINISHED_UNIT
23:13:54:I1:WU25:Core returned FINISHED_UNIT (100)
23:13:55:I1:Default:Added new work unit: cpus:0 gpus:gpu:01:00:00
23:13:55:I1:WU25:Uploading WU results
23:13:55:I1:WU27:Requesting WU assignment for user Nuon team 34099
23:13:55:I1:WU26:WARNING:Console control signal 1 on PID 4028
23:13:55:I1:WU26:Exiting, please wait. . .
23:13:56:I1:OUT8:> POST https://huangfolding1.chem.wisc.edu/api/results HTTP/1.1
23:13:56:I1:OUT9:> POST https://assign3.foldingathome.org/api/assign HTTP/1.1
23:13:56:I1:WU26:Folding@home Core Shutdown: INTERRUPTED
23:13:56:I1:WU26:Core returned INTERRUPTED (102)
23:13:56:I3:Running FahCore: C:\ProgramData\FAHClient\cores/fahcore-a8-win-64bit-avx2_256-0.0.12/FahCore_a8.exe -dir UFwOshvyxcP18Loo9__wYnntwqIgBJkS05MheiFmqZc -suffix 01 -version 8.4.9 -lifeline 15484 -np 21
23:13:56:I3:WU26:Started FahCore on PID 7392
23:13:56:I1:OUT9:< HTTP/1.1 200 HTTP_OK
23:13:56:I1:WU27:Received WU assignment 2-nNFd9euDC9Gm9_Vz6RYH1tKkjhCMa3cFMFMk3tHT8
23:13:56:I1:WU27:Downloading WU 
23:13:56:I1:WU26:*********************** Log Started 2024-12-01T23:13:56Z ***********************


00:11:56:I1:WU26:Completed 4500000 out of 5000000 steps (90%)
00:12:25:I1:WU27:Completed 2475000 out of 2500000 steps (99%)
00:12:59:I1:WU27:Completed 2500000 out of 2500000 steps (100%)
00:12:59:I1:WU27:Average performance: 250.435 ns/day
00:12:59:I1:WU27:Checkpoint completed at step 2500000
00:13:05:I1:WU27:Saving result file ..\logfile_01.txt
00:13:05:I1:WU27:Saving result file checkpointIntegrator.xml
00:13:05:I1:WU27:Saving result file checkpointState.xml.bz2
00:13:05:I1:WU27:Saving result file positions.xtc
00:13:05:I1:WU27:Saving result file science.log
00:13:05:I1:WU27:Saving result file xtcAtoms.csv.bz2
00:13:05:I1:WU27:Folding@home Core Shutdown: FINISHED_UNIT
00:13:05:I1:WU27:Core returned FINISHED_UNIT (100)
00:13:05:I1:Default:Added new work unit: cpus:0 gpus:gpu:01:00:00
00:13:05:I1:WU27:Uploading WU results
00:13:05:I1:WU28:Requesting WU assignment for user Nuon team 34099
00:13:05:I1:WU26:WARNING:Console control signal 1 on PID 16736
00:13:05:I1:WU26:Exiting, please wait. . .
00:13:05:I1:WU26:Folding@home Core Shutdown: INTERRUPTED
00:13:06:I1:OUT11:> POST https://highland1.seas.upenn.edu/api/results HTTP/1.1
00:13:06:I1:OUT12:> POST https://assign4.foldingathome.org/api/assign HTTP/1.1
00:13:06:I1:OUT12:< HTTP/1.1 200 HTTP_OK
00:13:06:I1:WU28:Received WU assignment BbZa3kVOsdBpgHiwnVEJa8dUsLaQb4sxIRHKIOe3JDQ
00:13:06:I1:WU28:Downloading WU
00:13:06:I1:WU26:Core returned INTERRUPTED (102)
00:13:06:I3:Running FahCore: C:\ProgramData\FAHClient\cores/fahcore-a8-win-64bit-avx2_256-0.0.12/FahCore_a8.exe -dir UFwOshvyxcP18Loo9__wYnntwqIgBJkS05MheiFmqZc -suffix 01 -version 8.4.9 -lifeline 15484 -np 20
00:13:06:I3:WU26:Started FahCore on PID 17512
00:13:06:I1:OUT13:> POST https://highland2.seas.upenn.edu/api/assign HTTP/1.1
00:13:06:I1:WU26:*********************** Log Started 2024-12-02T00:13:06Z *********************** 


01:04:09:I1:WU29:Completed 1000000 out of 5000000 steps (20%)
01:04:31:I1:WU28:Completed 1237500 out of 1250000 steps (99%)
01:05:01:I1:WU28:Completed 1250000 out of 1250000 steps (100%)
01:05:01:I1:WU28:Average performance: 141.639 ns/day
01:05:02:I1:WU28:Checkpoint completed at step 1250000
01:05:12:I1:WU28:Saving result file ..\logfile_01.txt
01:05:12:I1:WU28:Saving result file checkpointIntegrator.xml
01:05:12:I1:WU28:Saving result file checkpointState.xml.bz2
01:05:12:I1:WU28:Saving result file positions.xtc
01:05:12:I1:WU28:Saving result file science.log
01:05:12:I1:WU28:Saving result file xtcAtoms.csv.bz2
01:05:12:I1:WU28:Folding@home Core Shutdown: FINISHED_UNIT
01:05:13:I1:WU28:Core returned FINISHED_UNIT (100)
01:05:13:I1:Default:Added new work unit: cpus:0 gpus:gpu:01:00:00
01:05:13:I1:WU28:Uploading WU results
01:05:13:I1:WU30:Requesting WU assignment for user Nuon team 34099
01:05:13:I1:WU29:WARNING:Console control signal 1 on PID 18252
01:05:13:I1:WU29:Exiting, please wait. . .
01:05:14:I1:WU29:Folding@home Core Shutdown: INTERRUPTED
01:05:14:I1:OUT17:> POST https://highland2.seas.upenn.edu/api/results HTTP/1.1
01:05:14:I1:OUT18:> POST https://assign6.foldingathome.org/api/assign HTTP/1.1
01:05:14:I1:OUT18:< HTTP/1.1 200 HTTP_OK
01:05:14:I1:WU30:Received WU assignment Atit5RoU1pIs017o0S-VOVVd0IaUJJUQwBSGy74X1Gw
01:05:14:I1:WU30:Downloading WU
01:05:14:I1:WU29:Core returned INTERRUPTED (102)
01:05:14:I3:Running FahCore: C:\ProgramData\FAHClient\cores/fahcore-a8-win-64bit-avx2_256-0.0.12/FahCore_a8.exe -dir HFLmVoe9COWo7dSF_mUaHXVLqwoWvybo4mOJhjrriYE -suffix 01 -version 8.4.9 -lifeline 15484 -np 20
01:05:14:I3:WU29:Started FahCore on PID 10244
01:05:14:I1:OUT19:> POST https://highland4.seas.upenn.edu/api/assign HTTP/1.1
01:05:14:I1:WU29:*********************** Log Started 2024-12-02T01:05:14Z *********************** 


02:09:07:I1:WU29:Completed 3450000 out of 5000000 steps (69%)
02:09:08:I1:WU30:Completed 1237500 out of 1250000 steps (99%)
02:09:46:I1:WU30:Completed 1250000 out of 1250000 steps (100%)
02:09:46:I1:WU30:Average performance: 113.684 ns/day
02:09:47:I1:WU30:Checkpoint completed at step 1250000
02:10:00:I1:WU30:Saving result file ..\logfile_01.txt
02:10:00:I1:WU30:Saving result file checkpointIntegrator.xml
02:10:00:I1:WU30:Saving result file checkpointState.xml.bz2
02:10:00:I1:WU30:Saving result file positions.xtc
02:10:00:I1:WU30:Saving result file science.log
02:10:00:I1:WU30:Saving result file xtcAtoms.csv.bz2
02:10:00:I1:WU30:Folding@home Core Shutdown: FINISHED_UNIT
02:10:00:I1:WU30:Core returned FINISHED_UNIT (100)
02:10:01:I1:Default:Added new work unit: cpus:0 gpus:gpu:01:00:00
02:10:01:I1:WU30:Uploading WU results
02:10:01:I1:WU31:Requesting WU assignment for user Nuon team 34099
02:10:01:I1:WU29:WARNING:Console control signal 1 on PID 10244
02:10:01:I1:WU29:Exiting, please wait. . .
02:10:01:I1:WU29:Folding@home Core Shutdown: INTERRUPTED
02:10:01:I1:OUT20:> POST https://highland4.seas.upenn.edu/api/results HTTP/1.1
02:10:01:I1:OUT21:> POST https://assign1.foldingathome.org/api/assign HTTP/1.1
02:10:02:I1:OUT21:< HTTP/1.1 200 HTTP_OK
02:10:02:I1:WU31:Received WU assignment zTXz3BSuagg7uIepat4QHhOAl_OpRp2mKAg7HQa6mh8
02:10:02:I1:WU31:Downloading WU
02:10:02:I1:WU29:Core returned INTERRUPTED (102)
02:10:02:I3:Running FahCore: C:\ProgramData\FAHClient\cores/fahcore-a8-win-64bit-avx2_256-0.0.12/FahCore_a8.exe -dir HFLmVoe9COWo7dSF_mUaHXVLqwoWvybo4mOJhjrriYE -suffix 01 -version 8.4.9 -lifeline 15484 -np 20
02:10:02:I3:WU29:Started FahCore on PID 21384
02:10:02:I1:OUT22:> POST https://highland1.seas.upenn.edu/api/assign HTTP/1.1
02:10:02:I1:WU29:*********************** Log Started 2024-12-02T02:10:02Z *********************** 


03:04:14:I1:WU31:Completed 2475000 out of 2500000 steps (99%)
03:04:20:I1:WU32:Completed 650000 out of 5000000 steps (13%)
03:04:47:I1:WU31:Completed 2500000 out of 2500000 steps (100%)
03:04:47:I1:WU31:Average performance: 261.818 ns/day
03:04:47:I1:WU31:Checkpoint completed at step 2500000
03:04:53:I1:WU31:Saving result file ..\logfile_01.txt
03:04:53:I1:WU31:Saving result file checkpointIntegrator.xml
03:04:53:I1:WU31:Saving result file checkpointState.xml.bz2
03:04:53:I1:WU31:Saving result file positions.xtc
03:04:53:I1:WU31:Saving result file science.log
03:04:53:I1:WU31:Saving result file xtcAtoms.csv.bz2
03:04:53:I1:WU31:Folding@home Core Shutdown: FINISHED_UNIT
03:04:54:I1:WU31:Core returned FINISHED_UNIT (100)
03:04:54:I1:Default:Added new work unit: cpus:0 gpus:gpu:01:00:00
03:04:54:I1:WU31:Uploading WU results
03:04:54:I1:WU33:Requesting WU assignment for user Nuon team 34099
03:04:54:I1:WU32:WARNING:Console control signal 1 on PID 7080
03:04:54:I1:WU32:Exiting, please wait. . .
03:04:54:I1:WU32:Folding@home Core Shutdown: INTERRUPTED
03:04:54:I1:OUT26:> POST https://highland1.seas.upenn.edu/api/results HTTP/1.1
03:04:54:I1:OUT27:> POST https://assign3.foldingathome.org/api/assign HTTP/1.1
03:04:55:I1:OUT27:< HTTP/1.1 200 HTTP_OK
03:04:55:I1:WU33:Received WU assignment yTTrOfnJVfN-w4ilEay1GB1LfiqFaGskLVZE6V-pMWs
03:04:55:I1:WU33:Downloading WU
03:04:55:I1:WU32:Core returned INTERRUPTED (102)
03:04:55:I3:Running FahCore: C:\ProgramData\FAHClient\cores/fahcore-a8-win-64bit-avx2_256-0.0.12/FahCore_a8.exe -dir QIb8D1lvl_rRPgdV17HKVRJ2I3T1WqPs1L7SwbulpRM -suffix 01 -version 8.4.9 -lifeline 15484 -np 20
03:04:55:I3:WU32:Started FahCore on PID 8960
03:04:55:I1:OUT28:> POST https://highland2.seas.upenn.edu/api/assign HTTP/1.1
03:04:55:I1:WU32:*********************** Log Started 2024-12-02T03:04:55Z ***********************
A minor problem with this version is that in the machine log the warnings check box is not working, when manually searching for warnings it finds warnings, with the checkbox not.
Top
calxalot
Site Moderator
Posts: 1273
Joined: Sat Dec 08, 2007 1:33 am
Location: San Francisco, CA
Contact:
Contact calxalot

Re: When assigning new CPU WU, GPU WU is interrupted

Post by calxalot »

Yes, they tend to interrupt each other as available cpus fluctuates.

Set folding to finish.
Wait for gpu work to finish.
In machine settings,
- disable the gpu
- click the lock at lower right
- create a separate resource group for your GPU, optionally with zero cpus

See also the guide:
https://foldingathome.org/v8-3-client-guide/
Top
Nuon
Posts: 3
Joined: Sun Nov 24, 2024 1:03 pm

Re: When assigning new CPU WU, GPU WU is interrupted

Post by Nuon »

Thank you for this workaround!
It helped me to nearly double the hourly points throughput. :D
Top
Post Reply

Return to “v8.4.xx Open Beta”