Resizable BAR

[Foxter]

Hello

Does anybody knows if Resizable BAR also improve the performance of GPU Folding@Home?

The Nvidia website description only mentions gaming improvements.

I am asking because I take care of some F@H platforms with Intel Z490 motherboards and RTX 3000 Series video cards and I want to know if it's worth the trouble of updating the motherboard and video cards BIOS just for F@H (these are F@H dedicated PCs, never used for gaming).

[iero]

In another forum (on a benchmarking thread), I've seen guys mention that the only measurable difference from BAR shows itself at 1440p ultra and above. (for exp. Horizon Dawn 1440p Ultra 102-->103fps / 4K Ultra 56--->63fp). I'll ask if it produces a different result in synthetic benchmarks and even if someone is willing to test BAR on F@H(doubtful).

[Gnomuz]

If I got it well, resizable BAR allows the CPU to access the whole graphics memory instead of small "chunks" at a time, which should speed up the big transfers of data into the graphics memory. Textures loading during gaming is a good example.
As for F@H, it has never been a graphics memory hog (!), generally I see between 300 and 450 MB of memory usage on my GPUs, depending on the project. So, I think there's very little to expect from resizable BAR for folding, but I may have misunderstood something, technical experts in GPU computing will correct if I'm wrong.

[iero]

If I got it well, resizable BAR allows the CPU to access the whole graphics memory instead of small "chunks" at a time, which should speed up the big transfers of data into the graphics memory. Textures loading during gaming is a good example.
As for F@H, it has never been a graphics memory hog (!), generally I see between 300 and 450 MB of memory usage on my GPUs, depending on the project. So, I think there's very little to expect from resizable BAR for folding, but I may have misunderstood something, technical experts in GPU computing will correct if I'm wrong.

You are not wrong, but given the faster transfer of data due to BAR, even if F@h uses low amounts of VRam, wouldn't reason have it that said speed would positively effect the outcome?

[JimboPalmer]

Both the initial load and unload of the WU would speed up and perhaps more importantly the checkpoint upload downloads would benefit.
However:
Only certain GPUs are supported
Only certain CPUs are supported
Only certain BIOSs are supported
Then you can write specific code to speed up the WU.

As a 'trailing edge' folder I would not expect to meet those spec this decade, but many users would buy that hardware if the entire supply wasn't bought up by crypto coin 'folders'.
Given the single developer F@H has, I would bet it will be awhile until he gets to this project.

[bruce]

Given the single developer F@H has, I would bet it will be awhile until he gets to this project.

While it sounds like a laudable idea, it would save a small percentage of run-time on a small percentage of hardware. It's a level of optimization that's not likely to happen.

The adjustment is aimed at frame rates in games. What PCIe utilization rate are you seeing in FAH and what is the rate for those designated games that are seeing a 10% improvement? Does that apply to 1x slot connections or 16x?

[MeeLee]

It was my impression that the bar just allows access for more VRAM memory?
Since FAH uses mostly below 500MB of VRAM, I don't think adjusting the bar values will improve performance.
The gaming performance improvements, are typical for accessing larger memory banks, which won't affect FAH by much.
If I'm not mistaken, FAH (used to, not sure if they still do) fold mostly at 800x600 pix.
A lower bar actually saves resources. A higher bar should in theory have some sort of performance penalty on low VRAM applications.

[Gnomuz]

A small follow-up on this subject, now that Nvidia has released a firmware update tool for their Founders Edition cards.

I've upgraded the BIOS of the motherboard (MSI MEG X570 Unify) to beta version 7C35vA91 (AGESA 1.2.0.1) with support for resizable BAR, and activated resizable BAR support in the BIOS settings.
I've updated the VBIOS of the RTX3060Ti with the Nvidia tool (Windows only tool, btw ...)
The machine dual boots Windows10 and Ubuntu, and the 3060Ti runs on a PCIe 4.0 8x (16 GB/s) slot.

Under Windows, resizable BAR was activated properly with the latest drivers 465.89, according to the Nvidia Control Panel.
Under Ubuntu the latest stable drivers were already installed (460.56).

While folding a WU of Project 17340, the memory-related output of 'nvidia-smi -q' under Ubuntu is :

Code: Select all

    FB Memory Usage
        Total                             : 7979 MiB
        Used                              : 560 MiB
        Free                              : 7419 MiB
    BAR1 Memory Usage
        Total                             : 8192 MiB
        Used                              : 12 MiB
        Free                              : 8180 MiB

I conclude resizable BAR is active under Ubuntu without a new driver, as BAR1 Memory total used to be 256 MiB before BIOS and VBIOS updates.

As it was only a snapshot, I've monitored the BAR Memory usage and PCIe bandwidth over 30 minutes while folding with 'nvidia dmon -s mt -c 1800'.
The synthetic result is :

Code: Select all

                                     Min          Max          Average
FB Memory Usage (MiB)                525          566          548
BAR1 Memory Usage (MiB)              12           12           12
PCIe Rx Throughput (MB/s)            0            4643         3898
PCIe Tx Throughput (MB/s)            0            1878         399

As it can be guessed, I don't see any difference in folding performance with resizable BAR activated, which, I think, can be explained by a constant and very low usage of BAR memory by F@H (12 MiB/8192 Mib), although the average Rx PCIe bandwdith (3.8 GB/s) is rather high at 24.3% of the theoretical bandwidth (16 GB/s) on this WU (project 17340).

[MeeLee]

12MB for VRAM seems unusually low. Especially for a 3060.
Performance should suffer tremendously, unless it's folding an extremely small atom count WU.
there's also something about min/max values being identical.
Configuration setting error?

If a large WU is being processed with only 12MB of VRAM, it more than likely will need to use a lot more PCIE bandwidth, to collect the needed data from RAM, and may even result in serious performance issues...
It just doesn't make sense to me, why it would only use 12MB.

[iero]

On my 3060 the 15day avg Vram usage is 1,05GB.

[Gnomuz]

Hi all,

I think there was some misunderstanding on my previous post.
The average VRAM (or FB Memory) usage (first line) was 548 MB (min 525, max 566) during my 30 minutes monitoring of a WU from project 17340, which is rather consistent with what I've seen on other projects. It may be more or less, depending on the atoms count of the WU.
But 12MB is the average, min and max BAR1memory usage (second line).

The memory related output of 'nvidia-smi -q' (command available on both Windows and Linux) currently looks like :

Code: Select all

    FB Memory Usage
        Total                             : 7979 MiB
        Used                              : 562 MiB
        Free                              : 7417 MiB
    BAR1 Memory Usage
        Total                             : 8192 MiB
        Used                              : 13 MiB
        Free                              : 8179 MiB

on my RTX3060Ti with 8GB of VRAM and resizable BAR enabled.
As already explained, the BAR1 total memory jumped from 256 MiB to 8192 MiB after enabling resizable BAR. If I got it well (see https://www.nvidia.com/en-us/geforce/ne ... r-support/), this feature aims at allowing the CPU to access the entire Frame Buffer (8 GB in my case) rather than accessing it through multiple small transfers (previously limited to 256 MB at a time), which should improve overall performance by optimizing CPU to GPU data transfers.
My understanding of my nvidia-smi output is the F@H client transfers at most 12 or 13 MB at a time, and thus doesn't benefit at all of a VRAM aperture (BAR1) size increase from 256MB to 8GB. Maybe a specific code optimization is needed to benefit from this feature, but that's far beyond my technical skills, and I'm sure others can elaborate.