FAH Control in Ubuntu 20.04

Moderators: Site Moderators, FAHC Science Team

justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

cfhdev wrote:Try

Code: Select all

sudo apt install ocl-icd-opencl-dev
Thanks so running that I got the following

Reading package lists... Done
Building dependency tree
Reading state information... Done
ocl-icd-opencl-dev is already the newest version (2.2.11-1ubuntu1).
The following package was automatically installed and is no longer required:
libnvidia-common-435
Use 'sudo apt autoremove' to remove it.
0 to upgrade, 0 to newly install, 0 to remove and 20 not to upgrade.

Code: Select all

*********************** Log Started 2020-05-26T01:44:21Z ***********************
01:44:21:Trying to access database...
01:44:21:Successfully acquired database lock
01:44:21:Downloading GPUs.txt from assign1.foldingathome.org:80
01:44:21:Connecting to assign1.foldingathome.org:80
01:44:22:Read GPUs.txt
01:44:22:Enabled folding slot 00: READY cpu:1
01:44:22:Enabled folding slot 01: READY gpu:0:GK208 [GeForce GT 710 LP]
01:44:23:ERROR:No compute devices matched GPU #0 {
01:44:23:ERROR:  "vendor": 4318,
01:44:23:ERROR:  "device": 4747,
01:44:23:ERROR:  "type": 2,
01:44:23:ERROR:  "species": 3,
01:44:23:ERROR:  "description": "GK208 [GeForce GT 710 LP]"
01:44:23:ERROR:}.  You may need to update your graphics drivers.
01:44:23:****************************** FAHClient ******************************
01:44:23:    Version: 7.6.13
01:44:23:     Author: Joseph Coffland <[email protected]>
01:44:23:  Copyright: 2020 foldingathome.org
01:44:23:   Homepage: https://foldingathome.org/
01:44:23:       Date: Apr 28 2020
01:44:23:       Time: 04:20:16
01:44:23:   Revision: 5a652817f46116b6e135503af97f18e094414e3b
01:44:23:     Branch: master
01:44:23:   Compiler: GNU 8.3.0
01:44:23:    Options: -std=c++11 -ffunction-sections -fdata-sections -O3 -funroll-loops
01:44:23:             -fno-pie
01:44:23:   Platform: linux2 4.19.0-5-amd64
01:44:23:       Bits: 64
01:44:23:       Mode: Release
01:44:23:       Args: --child /etc/fahclient/config.xml --run-as fahclient
01:44:23:             --pid-file=/var/run/fahclient.pid --daemon
01:44:23:     Config: /etc/fahclient/config.xml
01:44:23:******************************** CBang ********************************
01:44:23:       Date: Apr 25 2020
01:44:23:       Time: 00:07:53
01:44:23:   Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
01:44:23:     Branch: master
01:44:23:   Compiler: GNU 8.3.0
01:44:23:    Options: -std=c++11 -ffunction-sections -fdata-sections -O3 -funroll-loops
01:44:23:             -fno-pie -fPIC
01:44:23:   Platform: linux2 4.19.0-5-amd64
01:44:23:       Bits: 64
01:44:23:       Mode: Release
01:44:23:******************************* System ********************************
01:44:23:        CPU: AMD Ryzen 7 3700X 8-Core Processor
01:44:23:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
01:44:23:       CPUs: 2
01:44:23:     Memory: 3.78GiB
01:44:23:Free Memory: 369.09MiB
01:44:23:    Threads: POSIX_THREADS
01:44:23: OS Version: 5.4
01:44:23:Has Battery: false
01:44:23: On Battery: false
01:44:23: UTC Offset: 10
01:44:23:        PID: 53681
01:44:23:        CWD: /var/lib/fahclient
01:44:23:         OS: Linux 5.4.0-31-generic x86_64
01:44:23:    OS Arch: AMD64
01:44:23:       GPUs: 1
01:44:23:      GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:3 GK208 [GeForce GT 710 LP]
01:44:23:       CUDA: Not detected: cuInit() returned 101
01:44:23:     OpenCL: Not detected: clGetPlatformIDs() returned -1001
01:44:23:******************************* libFAH ********************************
01:44:23:       Date: Apr 15 2020
01:44:23:       Time: 21:43:24
01:44:23:   Revision: 216968bc7025029c841ed6e36e81a03a316890d3
01:44:23:     Branch: master
01:44:23:   Compiler: GNU 8.3.0
01:44:23:    Options: -std=c++11 -ffunction-sections -fdata-sections -O3 -funroll-loops
01:44:23:             -fno-pie
01:44:23:   Platform: linux2 4.19.0-5-amd64
01:44:23:       Bits: 64
01:44:23:       Mode: Release
01:44:23:***********************************************************************
01:44:23:<config>
01:44:23:  <!-- Client Control -->
01:44:23:  <fold-anon v='true'/>
01:44:23:
01:44:23:  <!-- Folding Slot Configuration -->
01:44:23:  <gpu v='false'/>
01:44:23:
01:44:23:  <!-- HTTP Server -->
01:44:23:  <allow v='127.0.0.1 192.168.1.230'/>
01:44:23:
01:44:23:  <!-- Network -->
01:44:23:  <proxy v=':8080'/>
01:44:23:
01:44:23:  <!-- Remote Command Server -->
01:44:23:  <password v='*****'/>
01:44:23:
01:44:23:  <!-- Slot Control -->
01:44:23:  <power v='FULL'/>
01:44:23:
01:44:23:  <!-- User Information -->
01:44:23:  <passkey v='*****'/>
01:44:23:  <team v='24'/>
01:44:23:  <user v='sic_vl2000'/>
01:44:23:
01:44:23:  <!-- Folding Slots -->
01:44:23:  <slot id='0' type='CPU'/>
01:44:23:  <slot id='1' type='GPU'/>
01:44:23:</config>
01:44:23:WU01:FS00:Starting
01:44:23:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/avx/Core_a7.fah/FahCore_a7 -dir 01 -suffix 01 -version 706 -lifeline 53681 -checkpoint 15 -np 1
01:44:23:WU01:FS00:Started FahCore on PID 53714
01:44:23:WU01:FS00:Core PID:53718
01:44:23:WU01:FS00:FahCore 0xa7 started
01:44:23:WU00:FS01:Cleaning up
01:44:23:WU00:FS01:Connecting to assign1.foldingathome.org:80
01:44:23:WU01:FS00:0xa7:*********************** Log Started 2020-05-26T01:44:23Z ***********************
01:44:23:WU01:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
01:44:23:WU01:FS00:0xa7:       Type: 0xa7
01:44:23:WU01:FS00:0xa7:       Core: Gromacs
01:44:23:WU01:FS00:0xa7:       Args: -dir 01 -suffix 01 -version 706 -lifeline 53714 -checkpoint 15 -np
01:44:23:WU01:FS00:0xa7:             1
01:44:23:WU01:FS00:0xa7:************************************ CBang *************************************
01:44:23:WU01:FS00:0xa7:       Date: Nov 5 2019
01:44:23:WU01:FS00:0xa7:       Time: 06:06:57
01:44:23:WU01:FS00:0xa7:   Revision: 46c96f1aa8419571d83f3e63f9c99a0d602f6da9
01:44:23:WU01:FS00:0xa7:     Branch: master
01:44:23:WU01:FS00:0xa7:   Compiler: GNU 8.3.0
01:44:23:WU01:FS00:0xa7:    Options: -std=c++11 -O3 -funroll-loops -fno-pie -fPIC
01:44:23:WU01:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
01:44:23:WU01:FS00:0xa7:       Bits: 64
01:44:23:WU01:FS00:0xa7:       Mode: Release
01:44:23:WU01:FS00:0xa7:************************************ System ************************************
01:44:23:WU01:FS00:0xa7:        CPU: AMD Ryzen 7 3700X 8-Core Processor
01:44:23:WU01:FS00:0xa7:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
01:44:23:WU01:FS00:0xa7:       CPUs: 2
01:44:23:WU01:FS00:0xa7:     Memory: 3.78GiB
01:44:23:WU01:FS00:0xa7:Free Memory: 355.99MiB
01:44:23:WU01:FS00:0xa7:    Threads: POSIX_THREADS
01:44:23:WU01:FS00:0xa7: OS Version: 5.4
01:44:23:WU01:FS00:0xa7:Has Battery: false
01:44:23:WU01:FS00:0xa7: On Battery: false
01:44:23:WU01:FS00:0xa7: UTC Offset: 10
01:44:23:WU01:FS00:0xa7:        PID: 53718
01:44:23:WU01:FS00:0xa7:        CWD: /var/lib/fahclient/work
01:44:23:WU01:FS00:0xa7:******************************** Build - libFAH ********************************
01:44:23:WU01:FS00:0xa7:    Version: 0.0.18
01:44:23:WU01:FS00:0xa7:     Author: Joseph Coffland <[email protected]>
01:44:23:WU01:FS00:0xa7:  Copyright: 2019 foldingathome.org
01:44:23:WU01:FS00:0xa7:   Homepage: https://foldingathome.org/
01:44:23:WU01:FS00:0xa7:       Date: Nov 5 2019
01:44:23:WU01:FS00:0xa7:       Time: 06:13:26
01:44:23:WU01:FS00:0xa7:   Revision: 490c9aa2957b725af319379424d5c5cb36efb656
01:44:23:WU01:FS00:0xa7:     Branch: master
01:44:23:WU01:FS00:0xa7:   Compiler: GNU 8.3.0
01:44:23:WU01:FS00:0xa7:    Options: -std=c++11 -O3 -funroll-loops -fno-pie
01:44:23:WU01:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
01:44:23:WU01:FS00:0xa7:       Bits: 64
01:44:23:WU01:FS00:0xa7:       Mode: Release
01:44:23:WU01:FS00:0xa7:************************************ Build *************************************
01:44:23:WU01:FS00:0xa7:       SIMD: avx_256
01:44:23:WU01:FS00:0xa7:********************************************************************************
01:44:23:WU01:FS00:0xa7:Project: 16424 (Run 1043, Clone 0, Gen 51)
01:44:23:WU01:FS00:0xa7:Unit: 0x0000003ea8f5c67d5e8f6916a59df321
01:44:23:WU01:FS00:0xa7:Digital signatures verified
01:44:23:WU01:FS00:0xa7:Calling: mdrun -s frame51.tpr -o frame51.trr -x frame51.xtc -cpi state.cpt -cpt 15 -nt 1
01:44:23:WU01:FS00:0xa7:Steps: first=12750000 total=250000
01:44:25:WU01:FS00:0xa7:Completed 206552 out of 250000 steps (82%)
01:44:27:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
01:44:27:WU00:FS01:Connecting to assign2.foldingathome.org:80
01:44:28:WU00:FS01:Assigned to work server 206.223.170.146
01:44:28:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GK208 [GeForce GT 710 LP] from 206.223.170.146
01:44:28:WU00:FS01:Connecting to 206.223.170.146:8080
01:44:40:WU00:FS01:Downloading 41.89MiB
01:44:46:WU00:FS01:Download 10.74%
01:44:52:WU00:FS01:Download 24.47%
01:44:58:WU00:FS01:Download 35.51%
01:45:04:WU00:FS01:Download 41.92%
01:45:10:WU00:FS01:Download 50.43%
01:45:16:WU00:FS01:Download 62.51%
01:45:22:WU00:FS01:Download 75.19%
01:45:28:WU00:FS01:Download 89.67%
01:45:32:WU00:FS01:Download complete
01:45:32:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14253 run:652 clone:4 gen:26 core:0x22 unit:0x0000002ccedfaa925eab700a6890e859
01:45:32:WU00:FS01:Starting
01:45:32:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
01:45:32:WU00:FS01:Starting
01:45:32:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
01:46:32:WU00:FS01:Starting
01:46:32:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
01:48:07:WU01:FS00:0xa7:Completed 207500 out of 250000 steps (83%)
01:48:10:WU00:FS01:Starting
01:48:10:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
I did also stop the FAH service and restarted it but haven't restarted the ubuntu box itself.
cfhdev
Posts: 41
Joined: Tue Mar 17, 2020 1:10 am

Re: FAH Control in Ubuntu 20.04

Post by cfhdev »

Reboot the system.
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

cfhdev wrote:Reboot the system.
Restarted and this is what it's getting

Code: Select all

02:00:21:WU00:FS00:FahCore 0xa7 started
02:00:21:WU00:FS00:0xa7:*********************** Log Started 2020-05-26T02:00:21Z ***********************
02:00:21:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
02:00:21:WU00:FS00:0xa7:       Type: 0xa7
02:00:21:WU00:FS00:0xa7:       Core: Gromacs
02:00:21:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 706 -lifeline 1367 -checkpoint 15 -np 1
02:00:21:WU00:FS00:0xa7:************************************ CBang *************************************
02:00:21:WU00:FS00:0xa7:       Date: Nov 5 2019
02:00:21:WU00:FS00:0xa7:       Time: 06:06:57
02:00:21:WU00:FS00:0xa7:   Revision: 46c96f1aa8419571d83f3e63f9c99a0d602f6da9
02:00:21:WU00:FS00:0xa7:     Branch: master
02:00:21:WU00:FS00:0xa7:   Compiler: GNU 8.3.0
02:00:21:WU00:FS00:0xa7:    Options: -std=c++11 -O3 -funroll-loops -fno-pie -fPIC
02:00:21:WU00:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
02:00:21:WU00:FS00:0xa7:       Bits: 64
02:00:21:WU00:FS00:0xa7:       Mode: Release
02:00:21:WU00:FS00:0xa7:************************************ System ************************************
02:00:21:WU00:FS00:0xa7:        CPU: AMD Ryzen 7 3700X 8-Core Processor
02:00:21:WU00:FS00:0xa7:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
02:00:21:WU00:FS00:0xa7:       CPUs: 2
02:00:21:WU00:FS00:0xa7:     Memory: 3.78GiB
02:00:21:WU00:FS00:0xa7:Free Memory: 2.65GiB
02:00:21:WU00:FS00:0xa7:    Threads: POSIX_THREADS
02:00:21:WU00:FS00:0xa7: OS Version: 5.4
02:00:21:WU00:FS00:0xa7:Has Battery: false
02:00:21:WU00:FS00:0xa7: On Battery: false
02:00:21:WU00:FS00:0xa7: UTC Offset: 10
02:00:21:WU00:FS00:0xa7:        PID: 1371
02:00:21:WU00:FS00:0xa7:        CWD: /var/lib/fahclient/work
02:00:21:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
02:00:21:WU00:FS00:0xa7:    Version: 0.0.18
02:00:21:WU00:FS00:0xa7:     Author: Joseph Coffland <[email protected]>
02:00:21:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
02:00:21:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
02:00:21:WU00:FS00:0xa7:       Date: Nov 5 2019
02:00:21:WU00:FS00:0xa7:       Time: 06:13:26
02:00:21:WU00:FS00:0xa7:   Revision: 490c9aa2957b725af319379424d5c5cb36efb656
02:00:21:WU00:FS00:0xa7:     Branch: master
02:00:21:WU00:FS00:0xa7:   Compiler: GNU 8.3.0
02:00:21:WU00:FS00:0xa7:    Options: -std=c++11 -O3 -funroll-loops -fno-pie
02:00:21:WU00:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
02:00:21:WU00:FS00:0xa7:       Bits: 64
02:00:21:WU00:FS00:0xa7:       Mode: Release
02:00:21:WU00:FS00:0xa7:************************************ Build *************************************
02:00:21:WU00:FS00:0xa7:       SIMD: avx_256
02:00:21:WU00:FS00:0xa7:********************************************************************************
02:00:21:WU00:FS00:0xa7:Project: 13851 (Run 0, Clone 16234, Gen 146)
02:00:21:WU00:FS00:0xa7:Unit: 0x000000ae287234c95e730175545e81ad
02:00:21:WU00:FS00:0xa7:Reading tar file core.xml
02:00:21:WU00:FS00:0xa7:Reading tar file frame146.tpr
02:00:21:WU00:FS00:0xa7:Digital signatures verified
02:00:21:WU00:FS00:0xa7:Calling: mdrun -s frame146.tpr -o frame146.trr -x frame146.xtc -e frame146.edr -cpt 15 -nt 1
02:00:21:WU00:FS00:0xa7:Steps: first=73000000 total=500000
02:00:21:WU00:FS00:0xa7:Completed 1 out of 500000 steps (0%)
02:00:22:WU01:FS00:Upload 67.24%
02:00:26:WU01:FS00:Upload complete
02:00:26:WU01:FS00:Server responded WORK_QUIT (404)
02:00:26:WARNING:WU01:FS00:Server did not like results, dumping
02:00:26:WU01:FS00:Cleaning up
02:01:16:WU02:FS01:Starting
02:01:16:ERROR:WU02:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
02:02:53:WU02:FS01:Starting
02:02:53:ERROR:WU02:FS01:Failed to start core: OpenCL device matching slot 1 not found, make sure the OpenCL driver is installed or try setting 'opencl-index' manually
cfhdev
Posts: 41
Joined: Tue Mar 17, 2020 1:10 am

Re: FAH Control in Ubuntu 20.04

Post by cfhdev »

Have you installed the Nvidia drivers?
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

cfhdev wrote:Have you installed the Nvidia drivers?
I get the following nvidia-smi
Unable to determine the device handle for GPU 0000:0B:00.0: Unknown Error

But if i go to software and settings > Additional drivers its set to Nvidia-driver-440
cfhdev
Posts: 41
Joined: Tue Mar 17, 2020 1:10 am

Re: FAH Control in Ubuntu 20.04

Post by cfhdev »

Can you post the results of

Code: Select all

nvidia-debugdump --list
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

Found 1 NVIDIA devices
Error: nvmlDeviceGetHandleByIndex(): Unknown Error
FAILED to get details on GPU (0x0): Unknown Error
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: FAH Control in Ubuntu 20.04

Post by bruce »

The GT 710 is a really old/slow device. While technically it is still supported, it won't accomplish much. I recommend you upgrade your GPU.
You can get a GTX 1050 and a GTX 960 that fit in most Low Profile slots but I don't remember if they fit in a single slot so if your PCIe slot is next to the case wall, you may be stuck.
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

bruce wrote:The GT 710 is a really old/slow device. While technically it is still supported, it won't accomplish much. I recommend you upgrade your GPU.
You can get a GTX 1050 and a GTX 960 that fit in most Low Profile slots but I don't remember if they fit in a single slot so if your PCIe slot is next to the case wall, you may be stuck.
Yeah, I know it is a shit card, it was only purchased when I was running a headless ESXi server, at the time I wasn't intending to do Folding etc. Now the use of the box has changed and will be looking at an RX 5500 XT that will be coming soon but was also going to try and use the GT in the meantime.
MeeLee
Posts: 1339
Joined: Tue Feb 19, 2019 10:16 pm

Re: FAH Control in Ubuntu 20.04

Post by MeeLee »

You may want to see if the Bios, Linux, or Windows recognizes the GPU correctly, and if the drivers are functioning fine, by installing a GPU benchmark program.
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

bruce wrote:The GT 710 is a really old/slow device. While technically it is still supported, it won't accomplish much. I recommend you upgrade your GPU.
You can get a GTX 1050 and a GTX 960 that fit in most Low Profile slots but I don't remember if they fit in a single slot so if your PCIe slot is next to the case wall, you may be stuck.
Yeah, I know it is a slow device it was only to be used on a headless ESXi hypervisor. I've been thinking though, could it be due to how my Ubuntu VM is passing through the video card ?
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

Looks like I sorted it, it appears I needed to make a change to my VM settings, for anyone that comes across the same issue this is what helped me.

https://tysonz.net/nvidia-gpu-passthrough-esxi-6-7
PantherX
Site Moderator
Posts: 6986
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Re: FAH Control in Ubuntu 20.04

Post by PantherX »

Did you need to use a headless plug or not?
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
justcruising
Posts: 13
Joined: Thu May 21, 2020 10:22 am

Re: FAH Control in Ubuntu 20.04

Post by justcruising »

PantherX wrote:Did you need to use a headless plug or not?
No, no headless plug needed, just make a change in the VM settings.
JimboPalmer
Posts: 2522
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: FAH Control in Ubuntu 20.04

Post by JimboPalmer »

if you need an Low Profile Single Slot card here is a GT 1030 LP

http://www.amazon.com/dp/B07D9MQDP4

Much better cards exist if you don't need Single Slot.

Double Wide Low Profile GTX 1650 www.amazon.com/dp/B081L3PSGK
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
Post Reply