OpenCL 1.2 finally in Mesa - please test

[Foliant]

As i noticed from the Phoronix-Newsletter OpenCL 1.2 should finally be supported by Mesa.
See the merge request: https://gitlab.freedesktop.org/mesa/mes ... uests/4974

I dont own any FAH-compatible AMD GPU so I cant test this for myself.
So if someone have the time and GPU - Im very interested if this will bring AMD to fold without ROCm and/or other packages/workarounds.


Regards
Patrick

[foldy]

If the mesa driver behaves the same as AMD driver then it should work with FAH. But if it behaves differently then FAHcore may fail. Then it would need fixes in Mesa driver or in FAHcore - and I don't know if FAH supports Mesa driver

[Foliant]

Im interested for two reasons.
First: Mesa possibly opens easy access to older AMD GPU and
Second: Maybe more users getting working clients by first trying FAH.

[Foliant]

So if someone have the time and GPU - Im very interested if this will bring AMD to fold without ROCm and/or other packages/workarounds.

I did get my hands on a

Code: Select all

$ FAHClient --lspci
VendorID:DeviceID:PCI Bus:PCI Slot:PCI function:Vendor Name:Description
0x1002:0x6798:1:0:0:Advanced Micro Devices, Inc. [AMD/ATI]:

This is listed as "1:5" in GPUs.txt:

0x1002:0x6798:1:5:Tahiti XT [Radeon R9 200/HD 7900/8970]

From my understanding this GPU should work.

On a freshly installed system, using Mesa 20.3 it wont even detect a usable GPU

Code: Select all

*********************** Log Started 2021-02-13T18:50:00Z ***********************
18:50:00:******************************* libFAH ********************************
18:50:00:       Date: Oct 20 2020
18:50:00:       Time: 20:36:39
18:50:00:   Revision: 5ca109d295a6245e2a2f590b3d0085ad5e567aeb
18:50:00:     Branch: master
18:50:00:   Compiler: GNU 8.3.0
18:50:00:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
18:50:00:             -fdata-sections -O3 -funroll-loops -fno-pie
18:50:00:   Platform: linux2 5.8.0-1-amd64
18:50:00:       Bits: 64
18:50:00:       Mode: Release
18:50:00:****************************** FAHClient ******************************
18:50:00:    Version: 7.6.21
18:50:00:     Author: Joseph Coffland <[email protected]>
18:50:00:  Copyright: 2020 foldingathome.org
18:50:00:   Homepage: https://foldingathome.org/
18:50:00:       Date: Oct 20 2020
18:50:00:       Time: 20:39:00
18:50:00:   Revision: 6efbf0e138e22d3963e6a291f78dcb9c6422a278
18:50:00:     Branch: master
18:50:00:   Compiler: GNU 8.3.0
18:50:00:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
18:50:00:             -fdata-sections -O3 -funroll-loops -fno-pie
18:50:00:   Platform: linux2 5.8.0-1-amd64
18:50:00:       Bits: 64
18:50:00:       Mode: Release
18:50:00:       Args: --child /etc/fahclient/config.xml --run-as fahclient
18:50:00:             --pid-file=/var/run/fahclient.pid --daemon
18:50:00:     Config: /etc/fahclient/config.xml
18:50:00:******************************** CBang ********************************
18:50:00:       Date: Oct 20 2020
18:50:00:       Time: 18:37:59
18:50:00:   Revision: 7e4ce85225d7eaeb775e87c31740181ca603de60
18:50:00:     Branch: master
18:50:00:   Compiler: GNU 8.3.0
18:50:00:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
18:50:00:             -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
18:50:00:   Platform: linux2 5.8.0-1-amd64
18:50:00:       Bits: 64
18:50:00:       Mode: Release
18:50:00:******************************* System ********************************
18:50:00:        CPU: Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz
18:50:00:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
18:50:00:       CPUs: 4
18:50:00:     Memory: 3.76GiB
18:50:00:Free Memory: 3.34GiB
18:50:00:    Threads: POSIX_THREADS
18:50:00: OS Version: 5.4
18:50:00:Has Battery: false
18:50:00: On Battery: false
18:50:00: UTC Offset: 1
18:50:00:        PID: 838
18:50:00:        CWD: /var/lib/fahclient
18:50:00:         OS: Linux 5.4.0-65-generic x86_64
18:50:00:    OS Arch: AMD64
18:50:00:       GPUs: 0
18:50:00:       CUDA: Not detected: Failed to open dynamic library 'libcuda.so':
18:50:00:             libcuda.so: cannot open shared object file: No such file or
18:50:00:             directory
18:50:00:     OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
18:50:00:             libOpenCL.so: cannot open shared object file: No such file or
18:50:00:             directory
18:50:00:***********************************************************************
18:50:00:<config>
18:50:00:  <!-- Client Control -->
18:50:00:  <fold-anon v='true'/>
18:50:00:
18:50:00:  <!-- Folding Slot Configuration -->
18:50:00:  <gpu v='false'/>
18:50:00:
18:50:00:  <!-- Network -->
18:50:00:  <proxy v=':8080'/>
18:50:00:
18:50:00:  <!-- User Information -->
18:50:00:  <passkey v='*****'/>
18:50:00:  <team v='******'/>
18:50:00:  <user v='******'/>
18:50:00:
18:50:00:  <!-- Folding Slots -->
18:50:00:  <slot id='0' type='CPU'>
18:50:00:    <cpus v='2'/>
18:50:00:    <paused v='True'/>
18:50:00:  </slot>
18:50:00:  <slot id='1' type='GPU'/>
18:50:00:</config>
18:50:00:Trying to access database...
18:50:00:Successfully acquired database lock
18:50:00:ERROR:Exception: No unallocated GPUs found
18:50:00:ERROR:Deleting slot 1
18:50:00:FS00:Initialized folding slot 00: cpu:2
18:50:00:WARNING:FS01:No CUDA or OpenCL 1.2+ support detected for GPU slot 01: gpu:-1:-1.  Disabling.
18:51:01:Saving configuration to /etc/fahclient/config.xml
18:51:01:<config>

This did not change after installing

Code: Select all

$ sudo apt install ocl-icd-opencl-dev

also no effect with

Code: Select all

$ sudo apt-get install libclc-amdgcn mesa-opencl-icd

So my dream having a driver that enables AMD users to fold without investing more time than just installing is busted.

As clinfo shows "OpenCL 1.1" my next step will be to try out ROCm.

Code: Select all

$ clinfo
Number of platforms                               1
  Platform Name                                   Clover
  Platform Vendor                                 Mesa
  Platform Version                                OpenCL 1.1 Mesa 20.3.4 - kisak-mesa PPA
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd
  Platform Extensions function suffix             MESA

  Platform Name                                   Clover
Number of devices                                 1
  Device Name                                     AMD TAHITI (DRM 2.50.0, 5.4.0-65-generic, LLVM 11.0.1)
  Device Vendor                                   AMD
  Device Vendor ID                                0x1002
  Device Version                                  OpenCL 1.1 Mesa 20.3.4 - kisak-mesa PPA
  Driver Version                                  20.3.4 - kisak-mesa PPA
  Device OpenCL C Version                         OpenCL C 1.1 
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Max compute units                               32
  Max clock frequency                             1050MHz
  Max work item dimensions                        3
  Max work item sizes                             256x256x256
  Max work group size                             256
  Preferred work group size multiple              64
  Preferred / native vector sizes                 
    char                                                16 / 16      
    short                                                8 / 8       
    int                                                  4 / 4       
    long                                                 2 / 2       
    half                                                 0 / 0        (n/a)
    float                                                4 / 4       
    double                                               2 / 2        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     No
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 No
    Round to infinity                             No
    IEEE754-2008 fused multiply-add               No
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              3221225472 (3GiB)
  Error Correction support                        No
  Max memory allocation                           2254857830 (2.1GiB)
  Unified memory for Host and Device              No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       32768 bits (4096 bytes)
  Global Memory cache type                        None
  Image support                                   No
  Local memory type                               Local
  Local memory size                               32768 (32KiB)
  Max number of constant args                     16
  Max constant buffer size                        67108864 (64MiB)
  Max size of kernel argument                     1024
  Queue properties                                
    Out-of-order execution                        No
    Profiling                                     Yes
  Profiling timer resolution                      0ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
  Device Extensions                               cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp64

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  Clover
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [MESA]
  clCreateContext(NULL, ...) [default]            Success [MESA]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
    Platform Name                                 Clover
    Device Name                                   AMD TAHITI (DRM 2.50.0, 5.4.0-65-generic, LLVM 11.0.1)
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  Success (1)
    Platform Name                                 Clover
    Device Name                                   AMD TAHITI (DRM 2.50.0, 5.4.0-65-generic, LLVM 11.0.1)
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
    Platform Name                                 Clover
    Device Name                                   AMD TAHITI (DRM 2.50.0, 5.4.0-65-generic, LLVM 11.0.1)

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.11
  ICD loader Profile                              OpenCL 2.1

Please stop me if the Radeon 7970 isnt supported at all.

Regards,
Patrick

[ajm]

Code: Select all

sudo apt install ocl-icd-opencl-dev

requires a reboot. Did you reboot after installing?

The only "Radeon 7970" in https://apps.foldingathome.org/GPUs.txt is

Code: Select all

0x1002:0x6800:1:5:Wimbledon XT [Radeon HD 7970M]

[JimboPalmer]

https://www.techpowerup.com/gpu-specs/r ... -7970.c296

As I understand it, there are two prerequisites to folding:
The GPU must support 64 bit Floating Point math (double precision) and yours does.
The GPU/driver should support OpenCL 1.2 and yours does. (some Nvidia GPUs only support OpenCL 1.1, but have all the OpenCL 1.2 parts Folding@Home needs, making it messy)
So with the right driver, your card should work.

Some GCN 1.0 cards don't work, but I think they lack 64 bit math.

If, as ajm says, it is not in GPUs.txt your will need to give us your GPU's device id, and I am not a Linux user so I can't say how to get that.

[bruce]

$ clinfo
Number of platforms 1
Platform Name Clover
Platform Vendor Mesa
Platform Version OpenCL 1.1 Mesa 20.3.4 -

OpenCL 1.1 is not supported at all. OpenCL 1.2 is the minimum required for FAH.

[Joe_H]

The only "Radeon 7970" in https://apps.foldingathome.org/GPUs.txt is
CODE: SELECT ALL
0x1002:0x6800:1:5:Wimbledon XT [Radeon HD 7970M]

For the desktop series of cards there are several entries that might match for the HD 7970:

Code: Select all

0x1002:0x6798:1:5:Tahiti XT [Radeon R9 200/HD 7900/8970]
0x1002:0x679a:1:5:Tahiti PRO [Radeon R9 280/HD 7900/8950]
0x1002:0x679b:1:5:Tahiti [Radeon HD 7900 Series]

Most likely is the first one. The detection is done by device ID, not name, and AMD uses the same ID number for a range of cards.

If you get Mesa to work, you will be the first.

As for ROCm, AMD only provided support for newer cards in that release. As far as I have seen, no one has done any back porting to add older cards like the HD 7970 using GCN 1st generation chips.

[Foliant]

Code: Select all

sudo apt install ocl-icd-opencl-dev

requires a reboot. Did you reboot after installing?

Yes.

If, as ajm says, it is not in GPUs.txt your will need to give us your GPU's device id, and I am not a Linux user so I can't say how to get that.

As you can see in my post above:

Code: Select all

$ FAHClient --lspci
VendorID:DeviceID:PCI Bus:PCI Slot:PCI function:Vendor Name:Description
0x1002:0x6798:1:0:0:Advanced Micro Devices, Inc. [AMD/ATI]:

This is listed as "1:5" in GPUs.txt:

0x1002:0x6798:1:5:Tahiti XT [Radeon R9 200/HD 7900/8970]

If you get Mesa to work, you will be the first.

I did read Mesa 20.3 did add support for OpenCL 1.2 and theyre even working on OpenCL 3.0 right now.
My idea was this could add support for "older cards".
As the 7970 supports OCL1.2 I wanted to try it.
The driver is showing support for 1.1 - that alone shouldnt happen with the new Mesa.

As for ROCm, AMD only provided support for newer cards in that release. As far as I have seen, no one has done any back porting to add older cards like the HD 7970 using GCN 1st generation chips.

Talking about GPUs I never was into the AMD-Universe before today.
And saddly youre right, I couldnt find a hint about my old GPU and ROCm.


[EDIT]Lookup the Specs: https://www.techpowerup.com/gpu-specs/x ... ition.b677 [/EDIT]

[Foliant]

Hmm... I fired up Win10, installed "adrenalin-2020-21.2.2-win10-64bit-feb11" and the GPU is also disabled.

Does AMD need "special treatment" in Windows as well?

Code: Select all

*********************** Log Started 2021-02-13T22:26:22Z ***********************
22:26:22:******************************* libFAH ********************************
22:26:22:           Date: Oct 20 2020
22:26:22:           Time: 13:36:55
22:26:22:       Revision: 5ca109d295a6245e2a2f590b3d0085ad5e567aeb
22:26:22:         Branch: master
22:26:22:       Compiler: Visual C++ 2015
22:26:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
22:26:22:       Platform: win32 10
22:26:22:           Bits: 32
22:26:22:           Mode: Release
22:26:22:****************************** FAHClient ******************************
22:26:22:        Version: 7.6.21
22:26:22:         Author: Joseph Coffland <[email protected]>
22:26:22:      Copyright: 2020 foldingathome.org
22:26:22:       Homepage: https://foldingathome.org/
22:26:22:           Date: Oct 20 2020
22:26:22:           Time: 13:41:04
22:26:22:       Revision: 6efbf0e138e22d3963e6a291f78dcb9c6422a278
22:26:22:         Branch: master
22:26:22:       Compiler: Visual C++ 2015
22:26:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
22:26:22:       Platform: win32 10
22:26:22:           Bits: 32
22:26:22:           Mode: Release
22:26:22:         Config: C:\ProgramData\FAHClient\config.xml
22:26:22:******************************** CBang ********************************
22:26:22:           Date: Oct 20 2020
22:26:22:           Time: 11:36:18
22:26:22:       Revision: 7e4ce85225d7eaeb775e87c31740181ca603de60
22:26:22:         Branch: master
22:26:22:       Compiler: Visual C++ 2015
22:26:22:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
22:26:22:       Platform: win32 10
22:26:22:           Bits: 32
22:26:22:           Mode: Release
22:26:22:******************************* System ********************************
22:26:22:            CPU: Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz
22:26:22:         CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
22:26:22:           CPUs: 4
22:26:22:         Memory: 3.91GiB
22:26:22:    Free Memory: 2.02GiB
22:26:22:        Threads: WINDOWS_THREADS
22:26:22:     OS Version: 6.2
22:26:22:    Has Battery: false
22:26:22:     On Battery: false
22:26:22:     UTC Offset: 1
22:26:22:            PID: 2672
22:26:22:            CWD: C:\ProgramData\FAHClient
22:26:22:  Win32 Service: false
22:26:22:             OS: Windows 10 Home
22:26:22:        OS Arch: AMD64
22:26:22:           GPUs: 1
22:26:22:          GPU 0: Bus:1 Slot:0 Func:0 AMD:1
22:26:22:           CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll': Das
22:26:22:                 angegebene Modul wurde nicht gefunden.
22:26:22:
22:26:22:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:3188.4
22:26:22:***********************************************************************
22:26:22:<config>
22:26:22:  <!-- Network -->
22:26:22:  <proxy v=':8080'/>
22:26:22:
22:26:22:  <!-- User Information -->
22:26:22:  <passkey v='*****'/>
22:26:22:  <team v='******'/>
22:26:22:  <user v='******'/>
22:26:22:
22:26:22:  <!-- Folding Slots -->
22:26:22:  <slot id='0' type='CPU'>
22:26:22:    <paused v='true'/>
22:26:22:  </slot>
22:26:22:  <slot id='1' type='GPU'>
22:26:22:    <pci-bus v='1'/>
22:26:22:    <pci-slot v='0'/>
22:26:22:  </slot>
22:26:22:</config>
22:26:22:Trying to access database...
22:26:22:Successfully acquired database lock
22:26:22:FS00:Initialized folding slot 00: cpu:2
22:26:22:WARNING:FS01:Disabling beta GPU slot 01: gpu:1:0.  Beta GPUs can be tested for no points by setting ``gpu-beta=true`` in the configuration.

[bruce]

If, as ajm says, it is not in GPUs.txt your will need to give us your GPU's device id, and I am not a Linux user so I can't say how to get that.

Two methods:
"FAHClient --lspci" in a text window (terminal) which has already been found
or
install lspci from your usual source of add-on programs

I did read Mesa 20.3 did add support for OpenCL 1.2 and theyre even working on OpenCL 3.0 right now.

OpenCL 3.0 may be advantageous for other apps, but the FAHCore is written to conform to the 1.2 API so that's what it will be using.

0x1002:0x6798:1:5:Tahiti XT [Radeon R9 200/HD 7900/8970]
is supported.

It should just be a matter of either manually installing OpenCL 1.2 or getting a version of Mesa that does that for you.

[PantherX]

Hmm... I fired up Win10, installed "adrenalin-2020-21.2.2-win10-64bit-feb11" and the GPU is also disabled.

Does AMD need "special treatment" in Windows as well? ...

Did your system download the latest version of GPUs.txt file? Assuming you had internet connection when FAHClient was started, it should have automatically download it and it would have worked unless; the GPUs.txt is corrupted or you didn't download the latest version of GPUs.txt file: viewtopic.php?f=80&t=36822

[bruce]

Hmm... I fired up Win10, installed "adrenalin-2020-21.2.2-win10-64bit-feb11" and the GPU is also disabled.

Does AMD need "special treatment" in Windows as well? ...

That depends on what you mean by "special treatment." As far as I know, GPU detection works the same way in Linux and Windows. (There have been recent changes to that code that I may not be familiar with.) It expects that if you're planning to fold with a GPU, the required drivers need to be preinstalled. I wouldn't call installing the correct drivers "special treatment" but it can be a complex process.

As PantnerX has said, the internet connection needs to be able to make unobstructed connections to the FAH servers the so FAHClient can continue setting up its files.

[Joe_H]

The only other thing I can think of is the driver. The latest driver you used may not have OpenCL support for older GCN 1st generation cards such as your HD 7970. AMD dropped much of its support for the older cards under Linux several years ago, video drivers just provide 2D and 3d acceleration, but not computation using OpenCL. There have been various issues reported in the past for some driver versions not working fully with older cards, so you may need to drop back a version or two with the Windows driver.

[ajm]

Quick test: I installed FAH 7.6.21 on a Win 10 laptop with 2 GPUs (Intel + Nvidia). Expert, start manually.
Then I opened C:\ProgramData\FAHClient\sample-config.xml :

Code: Select all

  <gpu value="false"/>
      <!-- If true, attempt to autoconfigure GPUs -->

Then I started FAH, configured the user, and let everything paused (default, as the laptop is on battery).
One GPU is configured, appears in FAHControl and there's no gpu v=false flag in the Expert tab.
Looks quite normal and healthy.