Page 1 of 3
I7 - FAH crashing, potentially due to Linux kernel upgrade?
Posted: Tue Sep 17, 2019 8:41 pm
by fangfufu
For the past several weeks, I noticed that my FAH instance isn't folding WU all the way to the end. It always mysteriously crashes. Right now, it is complaining about "BAD_FRAME_CHECKSUM". The only thing I can think of that might have caused the problem is that I am now running Debian Buster.
This is my Linux kernel's version:
Code: Select all
$ uname -a
Linux smithsonian 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2 (2019-08-28) x86_64 GNU/Linux
I did add the following as my Linux kernel command line option:
Otherwise it segfaults. Now it no longer segfaults, but WUs are not being completed properly.
This is the log of a WU that crashed:
Code: Select all
19:31:08:FS00:Unpaused
19:31:08:WU01:FS00:Starting
19:31:08:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 01 -suffix 01 -version 705 -lifeline 3977 -checkpoint 30 -np 2
19:31:08:WU01:FS00:Started FahCore on PID 20923
19:31:08:WU01:FS00:Core PID:20927
19:31:08:WU01:FS00:FahCore 0xa7 started
19:31:09:WU01:FS00:0xa7:*********************** Log Started 2019-09-17T19:31:09Z ***********************
19:31:09:WU01:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
19:31:09:WU01:FS00:0xa7: Type: 0xa7
19:31:09:WU01:FS00:0xa7: Core: Gromacs
19:31:09:WU01:FS00:0xa7: Website: https://foldingathome.org/
19:31:09:WU01:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
19:31:09:WU01:FS00:0xa7: Author: Joseph Coffland <[email protected]>
19:31:09:WU01:FS00:0xa7: Args: -dir 01 -suffix 01 -version 705 -lifeline 20923 -checkpoint 30 -np
19:31:09:WU01:FS00:0xa7: 2
19:31:09:WU01:FS00:0xa7: Config: <none>
19:31:09:WU01:FS00:0xa7:************************************ Build *************************************
19:31:09:WU01:FS00:0xa7: Version: 0.0.17
19:31:09:WU01:FS00:0xa7: Date: Apr 27 2018
19:31:09:WU01:FS00:0xa7: Time: 19:09:21
19:31:09:WU01:FS00:0xa7: Repository: Git
19:31:09:WU01:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
19:31:09:WU01:FS00:0xa7: Branch: master
19:31:09:WU01:FS00:0xa7: Compiler: GNU 6.3.0 20170516
19:31:09:WU01:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
19:31:09:WU01:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
19:31:09:WU01:FS00:0xa7: Bits: 64
19:31:09:WU01:FS00:0xa7: Mode: Release
19:31:09:WU01:FS00:0xa7: SIMD: avx_256
19:31:09:WU01:FS00:0xa7:************************************ System ************************************
19:31:09:WU01:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
19:31:09:WU01:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
19:31:09:WU01:FS00:0xa7: CPUs: 8
19:31:09:WU01:FS00:0xa7: Memory: 15.57GiB
19:31:09:WU01:FS00:0xa7:Free Memory: 1.63GiB
19:31:09:WU01:FS00:0xa7: Threads: POSIX_THREADS
19:31:09:WU01:FS00:0xa7: OS Version: 4.19
19:31:09:WU01:FS00:0xa7:Has Battery: true
19:31:09:WU01:FS00:0xa7: On Battery: false
19:31:09:WU01:FS00:0xa7: UTC Offset: 1
19:31:09:WU01:FS00:0xa7: PID: 20927
19:31:09:WU01:FS00:0xa7: CWD: /var/lib/fahclient/work
19:31:09:WU01:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
19:31:09:WU01:FS00:0xa7: OS Arch: AMD64
19:31:09:WU01:FS00:0xa7:********************************************************************************
19:31:09:WU01:FS00:0xa7:Project: 14189 (Run 4, Clone 43, Gen 4)
19:31:09:WU01:FS00:0xa7:Unit: 0x000000050002894b5d543ddcfc61fa52
19:31:09:WU01:FS00:0xa7:Digital signatures verified
19:31:09:WU01:FS00:0xa7:Calling: mdrun -s frame4.tpr -o frame4.trr -cpi state.cpt -cpt 30 -nt 2
19:31:09:WU01:FS00:0xa7:Steps: first=5000000 total=1250000
19:31:10:WU01:FS00:0xa7:Completed 68862 out of 1250000 steps (5%)
19:31:32:Removing old file 'configs/config-20190829-140746.xml'
19:31:32:Saving configuration to /etc/fahclient/config.xml
19:31:32:<config>
19:31:32: <!-- Client Control -->
19:31:32: <fold-anon v='true'/>
19:31:32:
19:31:32: <!-- Folding Core -->
19:31:32: <checkpoint v='30'/>
19:31:32:
19:31:32: <!-- Folding Slot Configuration -->
19:31:32: <gpu v='false'/>
19:31:32:
19:31:32: <!-- Network -->
19:31:32: <proxy v=':8080'/>
19:31:32:
19:31:32: <!-- User Information -->
19:31:32: <passkey v='********************************'/>
19:31:32: <team v='224497'/>
19:31:32: <user v='fangfufu'/>
19:31:32:
19:31:32: <!-- Folding Slots -->
19:31:32: <slot id='0' type='CPU'>
19:31:32: <cpus v='2'/>
19:31:32: </slot>
19:31:32:</config>
19:41:46:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
19:41:46:WU01:FS00:Starting
19:41:46:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 01 -suffix 01 -version 705 -lifeline 3977 -checkpoint 30 -np 2
19:41:46:WU01:FS00:Started FahCore on PID 9067
19:41:46:WU01:FS00:Core PID:9071
19:41:46:WU01:FS00:FahCore 0xa7 started
19:41:46:WARNING:WU01:FS00:FahCore returned: BAD_FRAME_CHECKSUM (112 = 0x70)
19:41:46:WARNING:WU01:FS00:Fatal error, dumping
19:41:46:WU01:FS00:Sending unit results: id:01 state:SEND error:DUMPED project:14189 run:4 clone:43 gen:4 core:0xa7 unit:0x000000050002894b5d543ddcfc61fa52
19:41:46:WU01:FS00:Connecting to 155.247.166.219:8080
19:41:47:WU00:FS00:Connecting to 65.254.110.245:8080
19:41:47:WU01:FS00:Server responded WORK_ACK (400)
19:41:47:WU01:FS00:Cleaning up
Below are all the error logs, obtained from FAHControl, after filtering by "Warnings & Errors". I can't post the full log, because that would be over the limit.
Code: Select all
13:30:09:WU00:FS00:0xa7:ERROR:
13:30:09:WU00:FS00:0xa7:ERROR:-------------------------------------------------------
13:30:09:WU00:FS00:0xa7:ERROR:Program GROMACS, VERSION 5.0.4-20161122-4846b12ba-unknown
13:30:09:WU00:FS00:0xa7:ERROR:Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/pme.c, line: 754
13:30:09:WU00:FS00:0xa7:ERROR:
13:30:09:WU00:FS00:0xa7:ERROR:Fatal error:
13:30:09:WU00:FS00:0xa7:ERROR:1 particles communicated to PME rank 1 are more than 2/3 times the cut-off out of the domain decomposition cell of their charge group in dimension x.
13:30:09:WU00:FS00:0xa7:ERROR:This usually means that your system is not well equilibrated.
13:30:09:WU00:FS00:0xa7:ERROR:For more information and tips for troubleshooting, please check the GROMACS
13:30:09:WU00:FS00:0xa7:ERROR:website at http://www.gromacs.org/Documentation/Errors
13:30:09:WU00:FS00:0xa7:ERROR:-------------------------------------------------------
13:30:15:WARNING:WU00:FS00:FahCore returned: BAD_FRAME_CHECKSUM (112 = 0x70)
13:30:15:WARNING:WU00:FS00:Fatal error, dumping
13:30:17:WARNING:WU00:FS00:Server did not like results, dumping
******************************* Date: 2019-09-17 *******************************
16:30:09:WU01:FS00:0xa7:ERROR:
16:30:09:WU01:FS00:0xa7:ERROR:-------------------------------------------------------
16:30:09:WU01:FS00:0xa7:ERROR:Program GROMACS, VERSION 5.0.4-20161122-4846b12ba-unknown
16:30:09:WU01:FS00:0xa7:ERROR:Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/pme.c, line: 754
16:30:09:WU01:FS00:0xa7:ERROR:
16:30:09:WU01:FS00:0xa7:ERROR:Fatal error:
16:30:09:WU01:FS00:0xa7:ERROR:1 particles communicated to PME rank 1 are more than 2/3 times the cut-off out of the domain decomposition cell of their charge group in dimension x.
16:30:09:WU01:FS00:0xa7:ERROR:This usually means that your system is not well equilibrated.
16:30:09:WU01:FS00:0xa7:ERROR:For more information and tips for troubleshooting, please check the GROMACS
16:30:09:WU01:FS00:0xa7:ERROR:website at http://www.gromacs.org/Documentation/Errors
16:30:09:WU01:FS00:0xa7:ERROR:-------------------------------------------------------
17:46:49:WARNING:WU01:FS00:Detected clock skew (1 hours 14 mins), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
19:41:46:WARNING:WU01:FS00:FahCore returned: BAD_FRAME_CHECKSUM (112 = 0x70)
19:41:46:WARNING:WU01:FS00:Fatal error, dumping
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Wed Sep 18, 2019 10:30 pm
by bruce
I can think of several possibilites but you didn't post enough information for me to choose one.
What CPU is detected when FAH starts? (it's about 20 lines from the very beginning of FAH's log ... after a line that says "**** System ****"
Which instruction sets are supported by that CPU?
Are they all supported by your OS?
The messsage
13:30:09:WU00:FS00:0xa7:ERROR:Fatal error:
13:30:09:WU00:FS00:0xa7:ERROR:1 particles communicated to PME rank 1 are more than 2/3 times the cut-off out of the domain decomposition cell of their charge group in dimension x.
does indicate the WU is corrupt. When did you FIRST get that message and what WU had been assigned?
That WU might have been corrupt when it was downloaded or it may have been corrupted by the error you're investigating.
Either way, it's time to dump that WU and see what happens immediately after starting a fresh WU.
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Wed Sep 18, 2019 11:58 pm
by fangfufu
bruce, in the first of the two logs I submitted, it says:
Code: Select all
19:31:09:WU01:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
I changed the checkpoint interval to 3 mins, it seems to have temporarily solved the problem. If the problem occurs again, could I submit the log via pastebin?
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Thu Sep 19, 2019 4:45 am
by bruce
Changing the number of CPUs will rearrange the dimensions of the domain decomposition cell. I don't think that changing the checkpoint interval will change them.
Although it's rarely an issue, you did get a warning message when you restarted FAH with a different number of CPUs.
Is your CPU reverting to a power-saving state ("sleeping") periodically?
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Thu Sep 19, 2019 6:19 am
by fangfufu
Okay, I have figured out how to change the CPU frequency governor and CPU maximum frequency for thermal control and noise reduction purposes recently. I don't know if that's causing the problem...
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Thu Sep 19, 2019 6:27 am
by fangfufu
Okay,I tried changing the CPU maximum frequency, and frequency governor, and suspending / restarting the machine, it is not causing any problem. I will keep an eye out.
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Thu Sep 19, 2019 11:57 am
by fangfufu
dmesg shows one instance of segfauling. I don't know how relevant this is.
Code: Select all
[632464.620155] FahCore_a7[24022]: segfault at 7f3033e08060 ip 000055ed1533f5d2 sp 00007f2f87ffcc20 error 4 in FahCore_a7[55ed14ff0000+12bd000]
[632464.620177] Code: 57 d2 c4 41 23 5a dd c4 c1 7a 2c d3 c4 41 73 59 ec c5 aa 2a ce c5 fa 5c c1 4c 63 ce c4 41 28 57 d2 c4 c1 6b 5a d5 c5 fa 2c c2 <c4> 01 7a 58 2c 8f c5 aa 2a ca c5 22 5c d9 48 63 f2 c5 7a 11 2f c4
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Thu Sep 19, 2019 12:13 pm
by fangfufu
I dumped the WU, now is has gone into a loop. It keeps saying "FahCore returned: INTERRUPTED (102 = 0x66)".
Code: Select all
12:09:36:WU00:FS00:Starting
12:09:36:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 7335 -checkpoint 3 -np 2
12:09:36:WU00:FS00:Started FahCore on PID 7344
12:09:36:WU00:FS00:Core PID:7348
12:09:36:WU00:FS00:FahCore 0xa7 started
12:09:36:WU00:FS00:0xa7:*********************** Log Started 2019-09-19T12:09:36Z ***********************
12:09:36:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
12:09:36:WU00:FS00:0xa7: Type: 0xa7
12:09:36:WU00:FS00:0xa7: Core: Gromacs
12:09:36:WU00:FS00:0xa7: Website: https://foldingathome.org/
12:09:36:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
12:09:36:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
12:09:36:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 7344 -checkpoint 3 -np 2
12:09:36:WU00:FS00:0xa7: Config: <none>
12:09:36:WU00:FS00:0xa7:************************************ Build *************************************
12:09:36:WU00:FS00:0xa7: Version: 0.0.17
12:09:36:WU00:FS00:0xa7: Date: Apr 27 2018
12:09:36:WU00:FS00:0xa7: Time: 19:09:21
12:09:36:WU00:FS00:0xa7: Repository: Git
12:09:36:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
12:09:36:WU00:FS00:0xa7: Branch: master
12:09:36:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
12:09:36:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
12:09:36:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
12:09:36:WU00:FS00:0xa7: Bits: 64
12:09:36:WU00:FS00:0xa7: Mode: Release
12:09:36:WU00:FS00:0xa7: SIMD: avx_256
12:09:36:WU00:FS00:0xa7:************************************ System ************************************
12:09:36:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
12:09:36:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
12:09:36:WU00:FS00:0xa7: CPUs: 8
12:09:36:WU00:FS00:0xa7: Memory: 15.57GiB
12:09:36:WU00:FS00:0xa7:Free Memory: 2.48GiB
12:09:36:WU00:FS00:0xa7: Threads: POSIX_THREADS
12:09:36:WU00:FS00:0xa7: OS Version: 4.19
12:09:36:WU00:FS00:0xa7:Has Battery: true
12:09:36:WU00:FS00:0xa7: On Battery: false
12:09:36:WU00:FS00:0xa7: UTC Offset: 1
12:09:36:WU00:FS00:0xa7: PID: 7348
12:09:36:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
12:09:36:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
12:09:36:WU00:FS00:0xa7: OS Arch: AMD64
12:09:36:WU00:FS00:0xa7:********************************************************************************
12:09:36:WU00:FS00:0xa7:Project: 13827 (Run 134, Clone 1, Gen 30)
12:09:36:WU00:FS00:0xa7:Unit: 0x0000002180fccb095c9f846f93052fb5
12:09:36:WU00:FS00:0xa7:Digital signatures verified
12:09:36:WU00:FS00:0xa7:Calling: mdrun -s frame30.tpr -o frame30.trr -x frame30.xtc -cpt 3 -nt 2
12:09:36:WU00:FS00:0xa7:Steps: first=3750000 total=125000
12:09:40:WU00:FS00:0xa7:Completed 1 out of 125000 steps (0%)
12:09:44:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
12:09:44:WU00:FS00:Starting
12:09:44:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 7335 -checkpoint 3 -np 2
12:09:44:WU00:FS00:Started FahCore on PID 9655
12:09:44:WU00:FS00:Core PID:9659
12:09:44:WU00:FS00:FahCore 0xa7 started
12:09:45:WU00:FS00:0xa7:*********************** Log Started 2019-09-19T12:09:44Z ***********************
12:09:45:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
12:09:45:WU00:FS00:0xa7: Type: 0xa7
12:09:45:WU00:FS00:0xa7: Core: Gromacs
12:09:45:WU00:FS00:0xa7: Website: https://foldingathome.org/
12:09:45:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
12:09:45:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
12:09:45:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 9655 -checkpoint 3 -np 2
12:09:45:WU00:FS00:0xa7: Config: <none>
12:09:45:WU00:FS00:0xa7:************************************ Build *************************************
12:09:45:WU00:FS00:0xa7: Version: 0.0.17
12:09:45:WU00:FS00:0xa7: Date: Apr 27 2018
12:09:45:WU00:FS00:0xa7: Time: 19:09:21
12:09:45:WU00:FS00:0xa7: Repository: Git
12:09:45:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
12:09:45:WU00:FS00:0xa7: Branch: master
12:09:45:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
12:09:45:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
12:09:45:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
12:09:45:WU00:FS00:0xa7: Bits: 64
12:09:45:WU00:FS00:0xa7: Mode: Release
12:09:45:WU00:FS00:0xa7: SIMD: avx_256
12:09:45:WU00:FS00:0xa7:************************************ System ************************************
12:09:45:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
12:09:45:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
12:09:45:WU00:FS00:0xa7: CPUs: 8
12:09:45:WU00:FS00:0xa7: Memory: 15.57GiB
12:09:45:WU00:FS00:0xa7:Free Memory: 2.55GiB
12:09:45:WU00:FS00:0xa7: Threads: POSIX_THREADS
12:09:45:WU00:FS00:0xa7: OS Version: 4.19
12:09:45:WU00:FS00:0xa7:Has Battery: true
12:09:45:WU00:FS00:0xa7: On Battery: false
12:09:45:WU00:FS00:0xa7: UTC Offset: 1
12:09:45:WU00:FS00:0xa7: PID: 9659
12:09:45:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
12:09:45:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
12:09:45:WU00:FS00:0xa7: OS Arch: AMD64
12:09:45:WU00:FS00:0xa7:********************************************************************************
12:09:45:WU00:FS00:0xa7:Project: 13827 (Run 134, Clone 1, Gen 30)
12:09:45:WU00:FS00:0xa7:Unit: 0x0000002180fccb095c9f846f93052fb5
12:09:45:WU00:FS00:0xa7:Digital signatures verified
12:09:45:WU00:FS00:0xa7:Calling: mdrun -s frame30.tpr -o frame30.trr -x frame30.xtc -cpt 3 -nt 2
12:09:45:WU00:FS00:0xa7:Steps: first=3750000 total=125000
12:09:49:WU00:FS00:0xa7:Completed 1 out of 125000 steps (0%)
12:09:49:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
12:10:44:WU00:FS00:Starting
12:10:44:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 7335 -checkpoint 3 -np 2
12:10:44:WU00:FS00:Started FahCore on PID 1269
12:10:44:WU00:FS00:Core PID:1273
12:10:44:WU00:FS00:FahCore 0xa7 started
12:10:45:WU00:FS00:0xa7:*********************** Log Started 2019-09-19T12:10:44Z ***********************
12:10:45:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
12:10:45:WU00:FS00:0xa7: Type: 0xa7
12:10:45:WU00:FS00:0xa7: Core: Gromacs
12:10:45:WU00:FS00:0xa7: Website: https://foldingathome.org/
12:10:45:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
12:10:45:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
12:10:45:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 1269 -checkpoint 3 -np 2
12:10:45:WU00:FS00:0xa7: Config: <none>
12:10:45:WU00:FS00:0xa7:************************************ Build *************************************
12:10:45:WU00:FS00:0xa7: Version: 0.0.17
12:10:45:WU00:FS00:0xa7: Date: Apr 27 2018
12:10:45:WU00:FS00:0xa7: Time: 19:09:21
12:10:45:WU00:FS00:0xa7: Repository: Git
12:10:45:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
12:10:45:WU00:FS00:0xa7: Branch: master
12:10:45:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
12:10:45:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
12:10:45:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
12:10:45:WU00:FS00:0xa7: Bits: 64
12:10:45:WU00:FS00:0xa7: Mode: Release
12:10:45:WU00:FS00:0xa7: SIMD: avx_256
12:10:45:WU00:FS00:0xa7:************************************ System ************************************
12:10:45:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
12:10:45:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
12:10:45:WU00:FS00:0xa7: CPUs: 8
12:10:45:WU00:FS00:0xa7: Memory: 15.57GiB
12:10:45:WU00:FS00:0xa7:Free Memory: 1.99GiB
12:10:45:WU00:FS00:0xa7: Threads: POSIX_THREADS
12:10:45:WU00:FS00:0xa7: OS Version: 4.19
12:10:45:WU00:FS00:0xa7:Has Battery: true
12:10:45:WU00:FS00:0xa7: On Battery: false
12:10:45:WU00:FS00:0xa7: UTC Offset: 1
12:10:45:WU00:FS00:0xa7: PID: 1273
12:10:45:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
12:10:45:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
12:10:45:WU00:FS00:0xa7: OS Arch: AMD64
12:10:45:WU00:FS00:0xa7:********************************************************************************
12:10:45:WU00:FS00:0xa7:Project: 13827 (Run 134, Clone 1, Gen 30)
12:10:45:WU00:FS00:0xa7:Unit: 0x0000002180fccb095c9f846f93052fb5
12:10:45:WU00:FS00:0xa7:Digital signatures verified
12:10:45:WU00:FS00:0xa7:Calling: mdrun -s frame30.tpr -o frame30.trr -x frame30.xtc -cpt 3 -nt 2
12:10:45:WU00:FS00:0xa7:Steps: first=3750000 total=125000
12:10:50:WU00:FS00:0xa7:Completed 1 out of 125000 steps (0%)
12:10:50:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
12:11:38:Removing old file 'configs/config-20190917-182527.xml'
12:11:38:Saving configuration to /etc/fahclient/config.xml
12:11:38:<config>
12:11:38: <!-- Client Control -->
12:11:38: <fold-anon v='true'/>
12:11:38:
12:11:38: <!-- Folding Core -->
12:11:38: <checkpoint v='3'/>
12:11:38:
12:11:38: <!-- Folding Slot Configuration -->
12:11:38: <gpu v='false'/>
12:11:38:
12:11:38: <!-- Network -->
12:11:38: <proxy v=':8080'/>
12:11:38:
12:11:38: <!-- Slot Control -->
12:11:38: <power v='medium'/>
12:11:38:
12:11:38: <!-- User Information -->
12:11:38: <passkey v='********************************'/>
12:11:38: <team v='224497'/>
12:11:38: <user v='fangfufu'/>
12:11:38:
12:11:38: <!-- Folding Slots -->
12:11:38: <slot id='0' type='CPU'>
12:11:38: <cpus v='2'/>
12:11:38: </slot>
12:11:38:</config>
12:11:44:WU00:FS00:Starting
12:11:44:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 7335 -checkpoint 3 -np 2
12:11:44:WU00:FS00:Started FahCore on PID 31243
12:11:44:WU00:FS00:Core PID:31248
12:11:44:WU00:FS00:FahCore 0xa7 started
12:11:45:WU00:FS00:0xa7:*********************** Log Started 2019-09-19T12:11:44Z ***********************
12:11:45:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
12:11:45:WU00:FS00:0xa7: Type: 0xa7
12:11:45:WU00:FS00:0xa7: Core: Gromacs
12:11:45:WU00:FS00:0xa7: Website: https://foldingathome.org/
12:11:45:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
12:11:45:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
12:11:45:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 31243 -checkpoint 3 -np 2
12:11:45:WU00:FS00:0xa7: Config: <none>
12:11:45:WU00:FS00:0xa7:************************************ Build *************************************
12:11:45:WU00:FS00:0xa7: Version: 0.0.17
12:11:45:WU00:FS00:0xa7: Date: Apr 27 2018
12:11:45:WU00:FS00:0xa7: Time: 19:09:21
12:11:45:WU00:FS00:0xa7: Repository: Git
12:11:45:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
12:11:45:WU00:FS00:0xa7: Branch: master
12:11:45:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
12:11:45:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
12:11:45:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
12:11:45:WU00:FS00:0xa7: Bits: 64
12:11:45:WU00:FS00:0xa7: Mode: Release
12:11:45:WU00:FS00:0xa7: SIMD: avx_256
12:11:45:WU00:FS00:0xa7:************************************ System ************************************
12:11:45:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
12:11:45:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
12:11:45:WU00:FS00:0xa7: CPUs: 8
12:11:45:WU00:FS00:0xa7: Memory: 15.57GiB
12:11:45:WU00:FS00:0xa7:Free Memory: 2.01GiB
12:11:45:WU00:FS00:0xa7: Threads: POSIX_THREADS
12:11:45:WU00:FS00:0xa7: OS Version: 4.19
12:11:45:WU00:FS00:0xa7:Has Battery: true
12:11:45:WU00:FS00:0xa7: On Battery: false
12:11:45:WU00:FS00:0xa7: UTC Offset: 1
12:11:45:WU00:FS00:0xa7: PID: 31248
12:11:45:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
12:11:45:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
12:11:45:WU00:FS00:0xa7: OS Arch: AMD64
12:11:45:WU00:FS00:0xa7:********************************************************************************
12:11:45:WU00:FS00:0xa7:Project: 13827 (Run 134, Clone 1, Gen 30)
12:11:45:WU00:FS00:0xa7:Unit: 0x0000002180fccb095c9f846f93052fb5
12:11:45:WU00:FS00:0xa7:Digital signatures verified
12:11:45:WU00:FS00:0xa7:Calling: mdrun -s frame30.tpr -o frame30.trr -x frame30.xtc -cpt 3 -nt 2
12:11:45:WU00:FS00:0xa7:Steps: first=3750000 total=125000
12:11:49:WU00:FS00:0xa7:Completed 1 out of 125000 steps (0%)
12:12:00:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Fri Sep 20, 2019 2:41 am
by bruce
Where is the log showing the download of the new WU?
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Fri Sep 20, 2019 7:35 am
by Frisa
im running exact same kernel version on my folding pc, even same cpu architecture( albeit desktop cpu, i7 4790k) without any problem
i'd recommend you checking the temperature of your cpu
if you upgrade from debian stretch to buster, id recommend you do a clean reinstall, upgrade to newer major release always been problematic, no matter windows or linux
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Fri Sep 20, 2019 7:59 am
by fangfufu
Okay guys, it works fine now. I feel that I didn't do anything... This is really odd. If problem re-occurs, I will post it here. It has just managed to successfully fold a unit!
Bruce, if I throttle down the CPU, would it cause a unit to fail to fold? This is the only new thing I have been doing recently.
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Fri Sep 20, 2019 7:04 pm
by bruce
I've never seen any information suggesting it could. Early versions of the CPU-based FAHCore would work on a 8087 (which was very, very slow by today's standards) and in the early days of the Pentium, code was incorporated that used SSE but That should work. A year or so a go, code was added for AVX, (ditto).
You might go to gromacs.org and see if anybody has reported problems with underclocking, FAHCore_a7
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Fri Sep 20, 2019 8:55 pm
by fangfufu
Right bruce, it is doing the interrupted loop again... I didn't throttle down my CPU or anything. This time I am including the downloading the WU part, as requested.
This is an absolute nightmare... I have been running FAH for 15 years now. I have never had so much problems... I feel like I am losing my marbles...
Code: Select all
20:50:59:Adding folding slot 00: READY cpu:2
20:50:59:Removing old file 'configs/config-20190919-115809.xml'
20:50:59:Saving configuration to /etc/fahclient/config.xml
20:50:59:<config>
20:50:59: <!-- Client Control -->
20:50:59: <fold-anon v='true'/>
20:50:59:
20:50:59: <!-- Folding Core -->
20:50:59: <checkpoint v='30'/>
20:50:59:
20:50:59: <!-- Folding Slot Configuration -->
20:50:59: <gpu v='false'/>
20:50:59:
20:50:59: <!-- Network -->
20:50:59: <proxy v=':8080'/>
20:50:59:
20:50:59: <!-- User Information -->
20:50:59: <passkey v='********************************'/>
20:50:59: <team v='224497'/>
20:50:59: <user v='fangfufu'/>
20:50:59:
20:50:59: <!-- Folding Slots -->
20:50:59: <slot id='0' type='CPU'>
20:50:59: <cpus v='2'/>
20:50:59: </slot>
20:50:59:</config>
20:50:59:WU00:FS00:Connecting to 65.254.110.245:8080
20:50:59:WU00:FS00:Assigned to work server 155.247.166.219
20:50:59:WU00:FS00:Requesting new work unit for slot 00: READY cpu:2 from 155.247.166.219
20:50:59:WU00:FS00:Connecting to 155.247.166.219:8080
20:51:00:WU00:FS00:Downloading 6.16MiB
20:51:06:WU00:FS00:Download 79.19%
20:51:08:WU00:FS00:Download complete
20:51:08:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14189 run:1 clone:254 gen:6 core:0xa7 unit:0x000000080002894b5d77e3fca8d467fc
20:51:08:WU00:FS00:Starting
20:51:08:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 30389 -checkpoint 30 -np 2
20:51:08:WU00:FS00:Started FahCore on PID 20485
20:51:08:WU00:FS00:Core PID:20489
20:51:08:WU00:FS00:FahCore 0xa7 started
20:51:08:WU00:FS00:0xa7:*********************** Log Started 2019-09-20T20:51:08Z ***********************
20:51:08:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:51:08:WU00:FS00:0xa7: Type: 0xa7
20:51:08:WU00:FS00:0xa7: Core: Gromacs
20:51:08:WU00:FS00:0xa7: Website: https://foldingathome.org/
20:51:08:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
20:51:08:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
20:51:08:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 20485 -checkpoint 30 -np
20:51:08:WU00:FS00:0xa7: 2
20:51:08:WU00:FS00:0xa7: Config: <none>
20:51:08:WU00:FS00:0xa7:************************************ Build *************************************
20:51:08:WU00:FS00:0xa7: Version: 0.0.17
20:51:08:WU00:FS00:0xa7: Date: Apr 27 2018
20:51:08:WU00:FS00:0xa7: Time: 19:09:21
20:51:08:WU00:FS00:0xa7: Repository: Git
20:51:08:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
20:51:08:WU00:FS00:0xa7: Branch: master
20:51:08:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
20:51:08:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
20:51:08:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
20:51:08:WU00:FS00:0xa7: Bits: 64
20:51:08:WU00:FS00:0xa7: Mode: Release
20:51:08:WU00:FS00:0xa7: SIMD: avx_256
20:51:08:WU00:FS00:0xa7:************************************ System ************************************
20:51:08:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
20:51:08:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
20:51:08:WU00:FS00:0xa7: CPUs: 8
20:51:08:WU00:FS00:0xa7: Memory: 15.57GiB
20:51:08:WU00:FS00:0xa7:Free Memory: 2.35GiB
20:51:08:WU00:FS00:0xa7: Threads: POSIX_THREADS
20:51:08:WU00:FS00:0xa7: OS Version: 4.19
20:51:08:WU00:FS00:0xa7:Has Battery: true
20:51:08:WU00:FS00:0xa7: On Battery: false
20:51:08:WU00:FS00:0xa7: UTC Offset: 1
20:51:08:WU00:FS00:0xa7: PID: 20489
20:51:08:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
20:51:08:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
20:51:08:WU00:FS00:0xa7: OS Arch: AMD64
20:51:08:WU00:FS00:0xa7:********************************************************************************
20:51:08:WU00:FS00:0xa7:Project: 14189 (Run 1, Clone 254, Gen 6)
20:51:08:WU00:FS00:0xa7:Unit: 0x000000080002894b5d77e3fca8d467fc
20:51:08:WU00:FS00:0xa7:Reading tar file core.xml
20:51:08:WU00:FS00:0xa7:Reading tar file frame6.tpr
20:51:08:WU00:FS00:0xa7:Digital signatures verified
20:51:08:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpt 30 -nt 2
20:51:08:WU00:FS00:0xa7:Steps: first=7500000 total=1250000
20:51:11:WU00:FS00:0xa7:Completed 1 out of 1250000 steps (0%)
20:51:13:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
20:51:13:WU00:FS00:Starting
20:51:13:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 30389 -checkpoint 30 -np 2
20:51:13:WU00:FS00:Started FahCore on PID 21688
20:51:13:WU00:FS00:Core PID:21692
20:51:13:WU00:FS00:FahCore 0xa7 started
20:51:13:WU00:FS00:0xa7:*********************** Log Started 2019-09-20T20:51:13Z ***********************
20:51:13:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:51:13:WU00:FS00:0xa7: Type: 0xa7
20:51:13:WU00:FS00:0xa7: Core: Gromacs
20:51:13:WU00:FS00:0xa7: Website: https://foldingathome.org/
20:51:13:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
20:51:13:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
20:51:13:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 21688 -checkpoint 30 -np
20:51:13:WU00:FS00:0xa7: 2
20:51:13:WU00:FS00:0xa7: Config: <none>
20:51:13:WU00:FS00:0xa7:************************************ Build *************************************
20:51:13:WU00:FS00:0xa7: Version: 0.0.17
20:51:13:WU00:FS00:0xa7: Date: Apr 27 2018
20:51:13:WU00:FS00:0xa7: Time: 19:09:21
20:51:13:WU00:FS00:0xa7: Repository: Git
20:51:13:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
20:51:13:WU00:FS00:0xa7: Branch: master
20:51:13:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
20:51:13:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
20:51:13:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
20:51:13:WU00:FS00:0xa7: Bits: 64
20:51:13:WU00:FS00:0xa7: Mode: Release
20:51:13:WU00:FS00:0xa7: SIMD: avx_256
20:51:13:WU00:FS00:0xa7:************************************ System ************************************
20:51:13:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
20:51:13:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
20:51:13:WU00:FS00:0xa7: CPUs: 8
20:51:13:WU00:FS00:0xa7: Memory: 15.57GiB
20:51:13:WU00:FS00:0xa7:Free Memory: 2.38GiB
20:51:13:WU00:FS00:0xa7: Threads: POSIX_THREADS
20:51:13:WU00:FS00:0xa7: OS Version: 4.19
20:51:13:WU00:FS00:0xa7:Has Battery: true
20:51:13:WU00:FS00:0xa7: On Battery: false
20:51:13:WU00:FS00:0xa7: UTC Offset: 1
20:51:13:WU00:FS00:0xa7: PID: 21692
20:51:13:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
20:51:13:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
20:51:13:WU00:FS00:0xa7: OS Arch: AMD64
20:51:13:WU00:FS00:0xa7:********************************************************************************
20:51:13:WU00:FS00:0xa7:Project: 14189 (Run 1, Clone 254, Gen 6)
20:51:13:WU00:FS00:0xa7:Unit: 0x000000080002894b5d77e3fca8d467fc
20:51:13:WU00:FS00:0xa7:Digital signatures verified
20:51:13:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpt 30 -nt 2
20:51:13:WU00:FS00:0xa7:Steps: first=7500000 total=1250000
20:51:16:WU00:FS00:0xa7:Completed 1 out of 1250000 steps (0%)
20:51:17:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
20:51:26:Removing old file 'configs/config-20190919-115904.xml'
20:51:26:Saving configuration to /etc/fahclient/config.xml
20:51:26:<config>
20:51:26: <!-- Client Control -->
20:51:26: <fold-anon v='true'/>
20:51:26:
20:51:26: <!-- Folding Core -->
20:51:26: <checkpoint v='30'/>
20:51:26:
20:51:26: <!-- Folding Slot Configuration -->
20:51:26: <gpu v='false'/>
20:51:26:
20:51:26: <!-- Network -->
20:51:26: <proxy v=':8080'/>
20:51:26:
20:51:26: <!-- User Information -->
20:51:26: <passkey v='********************************'/>
20:51:26: <team v='224497'/>
20:51:26: <user v='fangfufu'/>
20:51:26:
20:51:26: <!-- Folding Slots -->
20:51:26: <slot id='0' type='CPU'>
20:51:26: <cpus v='2'/>
20:51:26: </slot>
20:51:26:</config>
20:52:13:WU00:FS00:Starting
20:52:13:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 30389 -checkpoint 30 -np 2
20:52:13:WU00:FS00:Started FahCore on PID 2165
20:52:13:WU00:FS00:Core PID:2171
20:52:13:WU00:FS00:FahCore 0xa7 started
20:52:14:WU00:FS00:0xa7:*********************** Log Started 2019-09-20T20:52:13Z ***********************
20:52:14:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:52:14:WU00:FS00:0xa7: Type: 0xa7
20:52:14:WU00:FS00:0xa7: Core: Gromacs
20:52:14:WU00:FS00:0xa7: Website: https://foldingathome.org/
20:52:14:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
20:52:14:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
20:52:14:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 2165 -checkpoint 30 -np 2
20:52:14:WU00:FS00:0xa7: Config: <none>
20:52:14:WU00:FS00:0xa7:************************************ Build *************************************
20:52:14:WU00:FS00:0xa7: Version: 0.0.17
20:52:14:WU00:FS00:0xa7: Date: Apr 27 2018
20:52:14:WU00:FS00:0xa7: Time: 19:09:21
20:52:14:WU00:FS00:0xa7: Repository: Git
20:52:14:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
20:52:14:WU00:FS00:0xa7: Branch: master
20:52:14:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
20:52:14:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
20:52:14:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
20:52:14:WU00:FS00:0xa7: Bits: 64
20:52:14:WU00:FS00:0xa7: Mode: Release
20:52:14:WU00:FS00:0xa7: SIMD: avx_256
20:52:14:WU00:FS00:0xa7:************************************ System ************************************
20:52:14:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
20:52:14:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
20:52:14:WU00:FS00:0xa7: CPUs: 8
20:52:14:WU00:FS00:0xa7: Memory: 15.57GiB
20:52:14:WU00:FS00:0xa7:Free Memory: 2.15GiB
20:52:14:WU00:FS00:0xa7: Threads: POSIX_THREADS
20:52:14:WU00:FS00:0xa7: OS Version: 4.19
20:52:14:WU00:FS00:0xa7:Has Battery: true
20:52:14:WU00:FS00:0xa7: On Battery: false
20:52:14:WU00:FS00:0xa7: UTC Offset: 1
20:52:14:WU00:FS00:0xa7: PID: 2171
20:52:14:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
20:52:14:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
20:52:14:WU00:FS00:0xa7: OS Arch: AMD64
20:52:14:WU00:FS00:0xa7:********************************************************************************
20:52:14:WU00:FS00:0xa7:Project: 14189 (Run 1, Clone 254, Gen 6)
20:52:14:WU00:FS00:0xa7:Unit: 0x000000080002894b5d77e3fca8d467fc
20:52:14:WU00:FS00:0xa7:Digital signatures verified
20:52:14:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpt 30 -nt 2
20:52:14:WU00:FS00:0xa7:Steps: first=7500000 total=1250000
20:52:16:WU00:FS00:0xa7:Completed 1 out of 1250000 steps (0%)
20:52:24:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
20:53:13:WU00:FS00:Starting
20:53:13:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 30389 -checkpoint 30 -np 2
20:53:13:WU00:FS00:Started FahCore on PID 15645
20:53:13:WU00:FS00:Core PID:15649
20:53:13:WU00:FS00:FahCore 0xa7 started
20:53:14:WU00:FS00:0xa7:*********************** Log Started 2019-09-20T20:53:13Z ***********************
20:53:14:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:53:14:WU00:FS00:0xa7: Type: 0xa7
20:53:14:WU00:FS00:0xa7: Core: Gromacs
20:53:14:WU00:FS00:0xa7: Website: https://foldingathome.org/
20:53:14:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
20:53:14:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
20:53:14:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 15645 -checkpoint 30 -np
20:53:14:WU00:FS00:0xa7: 2
20:53:14:WU00:FS00:0xa7: Config: <none>
20:53:14:WU00:FS00:0xa7:************************************ Build *************************************
20:53:14:WU00:FS00:0xa7: Version: 0.0.17
20:53:14:WU00:FS00:0xa7: Date: Apr 27 2018
20:53:14:WU00:FS00:0xa7: Time: 19:09:21
20:53:14:WU00:FS00:0xa7: Repository: Git
20:53:14:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
20:53:14:WU00:FS00:0xa7: Branch: master
20:53:14:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
20:53:14:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
20:53:14:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
20:53:14:WU00:FS00:0xa7: Bits: 64
20:53:14:WU00:FS00:0xa7: Mode: Release
20:53:14:WU00:FS00:0xa7: SIMD: avx_256
20:53:14:WU00:FS00:0xa7:************************************ System ************************************
20:53:14:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
20:53:14:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
20:53:14:WU00:FS00:0xa7: CPUs: 8
20:53:14:WU00:FS00:0xa7: Memory: 15.57GiB
20:53:14:WU00:FS00:0xa7:Free Memory: 2.23GiB
20:53:14:WU00:FS00:0xa7: Threads: POSIX_THREADS
20:53:14:WU00:FS00:0xa7: OS Version: 4.19
20:53:14:WU00:FS00:0xa7:Has Battery: true
20:53:14:WU00:FS00:0xa7: On Battery: false
20:53:14:WU00:FS00:0xa7: UTC Offset: 1
20:53:14:WU00:FS00:0xa7: PID: 15649
20:53:14:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
20:53:14:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
20:53:14:WU00:FS00:0xa7: OS Arch: AMD64
20:53:14:WU00:FS00:0xa7:********************************************************************************
20:53:14:WU00:FS00:0xa7:Project: 14189 (Run 1, Clone 254, Gen 6)
20:53:14:WU00:FS00:0xa7:Unit: 0x000000080002894b5d77e3fca8d467fc
20:53:14:WU00:FS00:0xa7:Digital signatures verified
20:53:14:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpt 30 -nt 2
20:53:14:WU00:FS00:0xa7:Steps: first=7500000 total=1250000
20:53:16:WU00:FS00:0xa7:Completed 1 out of 1250000 steps (0%)
20:53:22:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
20:54:13:WU00:FS00:Starting
20:54:13:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/AVX/Core_a7.fah/FahCore_a7 -dir 00 -suffix 01 -version 705 -lifeline 30389 -checkpoint 30 -np 2
20:54:13:WU00:FS00:Started FahCore on PID 27917
20:54:13:WU00:FS00:Core PID:27921
20:54:13:WU00:FS00:FahCore 0xa7 started
20:54:14:WU00:FS00:0xa7:*********************** Log Started 2019-09-20T20:54:13Z ***********************
20:54:14:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:54:14:WU00:FS00:0xa7: Type: 0xa7
20:54:14:WU00:FS00:0xa7: Core: Gromacs
20:54:14:WU00:FS00:0xa7: Website: https://foldingathome.org/
20:54:14:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
20:54:14:WU00:FS00:0xa7: Author: Joseph Coffland <[email protected]>
20:54:14:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 705 -lifeline 27917 -checkpoint 30 -np
20:54:14:WU00:FS00:0xa7: 2
20:54:14:WU00:FS00:0xa7: Config: <none>
20:54:14:WU00:FS00:0xa7:************************************ Build *************************************
20:54:14:WU00:FS00:0xa7: Version: 0.0.17
20:54:14:WU00:FS00:0xa7: Date: Apr 27 2018
20:54:14:WU00:FS00:0xa7: Time: 19:09:21
20:54:14:WU00:FS00:0xa7: Repository: Git
20:54:14:WU00:FS00:0xa7: Revision: 21359963583d09ec2063ef946399441c4df4ccd7
20:54:14:WU00:FS00:0xa7: Branch: master
20:54:14:WU00:FS00:0xa7: Compiler: GNU 6.3.0 20170516
20:54:14:WU00:FS00:0xa7: Options: -std=gnu++98 -O3 -funroll-loops
20:54:14:WU00:FS00:0xa7: Platform: linux2 4.14.0-3-amd64
20:54:14:WU00:FS00:0xa7: Bits: 64
20:54:14:WU00:FS00:0xa7: Mode: Release
20:54:14:WU00:FS00:0xa7: SIMD: avx_256
20:54:14:WU00:FS00:0xa7:************************************ System ************************************
20:54:14:WU00:FS00:0xa7: CPU: Intel(R) Core(TM) i7-4900MQ CPU @ 2.80GHz
20:54:14:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
20:54:14:WU00:FS00:0xa7: CPUs: 8
20:54:14:WU00:FS00:0xa7: Memory: 15.57GiB
20:54:14:WU00:FS00:0xa7:Free Memory: 2.46GiB
20:54:14:WU00:FS00:0xa7: Threads: POSIX_THREADS
20:54:14:WU00:FS00:0xa7: OS Version: 4.19
20:54:14:WU00:FS00:0xa7:Has Battery: true
20:54:14:WU00:FS00:0xa7: On Battery: false
20:54:14:WU00:FS00:0xa7: UTC Offset: 1
20:54:14:WU00:FS00:0xa7: PID: 27921
20:54:14:WU00:FS00:0xa7: CWD: /var/lib/fahclient/work
20:54:14:WU00:FS00:0xa7: OS: Linux 4.19.0-6-amd64 x86_64
20:54:14:WU00:FS00:0xa7: OS Arch: AMD64
20:54:14:WU00:FS00:0xa7:********************************************************************************
20:54:14:WU00:FS00:0xa7:Project: 14189 (Run 1, Clone 254, Gen 6)
20:54:14:WU00:FS00:0xa7:Unit: 0x000000080002894b5d77e3fca8d467fc
20:54:14:WU00:FS00:0xa7:Digital signatures verified
20:54:14:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpt 30 -nt 2
20:54:14:WU00:FS00:0xa7:Steps: first=7500000 total=1250000
20:54:16:WU00:FS00:0xa7:Completed 1 out of 1250000 steps (0%)
20:54:25:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Sat Sep 21, 2019 4:11 pm
by toTOW
Try to start the core manually in a terminal, you'll get a more detailed report of what is happening.
Re: FAH keeps crashing, potentially due to Linux kernel upgr
Posted: Sat Sep 21, 2019 4:13 pm
by fangfufu
toTOW wrote:Try to start the core manually in a terminal, you'll get a more detailed report of what is happening.
How do I do that?