Run multiple instances of FAHClient on same linux computer

[PRP_R148H]

Following on from an ancient discussion here

Code: Select all

https://foldingforum.org/viewtopic.php?f=16&t=11372

, a user asks if it's possible to run 2 local clients on 1 computer. The answer comes here

Code: Select all

https://web.archive.org/web/20160315165828/http://fahwiki.net/index.php/FAH_%26_SMP#Running_Multiple_Clients

, where that wiki entry outlines how it's possible on windows to simply install the console in two different locations and run.

On linux, I'm having a hard time conceptualising how to run 2 clients (each with different config info) and have it controlled on the same FAHControl unit. I understand I can add (remote) clients to the Control unit, but for another local one need to run another FAHClient and expose it to a different port.

Any ideas? Can I run parallel FAHClient processes, or do I need to install two FAHClients in different locations. If so, how?

[Joe_H]

The problem and solution you found are related to a much older version of the folding client that predates the V7 client. The current folding client is intended to only have a single FAHClient process running that manages one or more folding slots for CPU or GPU folding.

In the past, a few did configure slots with different usernames and passkeys set. This did work then, I don't know if anyone has tested doing that with the current or recent V7 folding client installations. This would require using FAHControl to add the username and passkey information as Extra slot options to each GPU slot. If it still works, then each slot would download and upload WUs as being for different users.

[PRP_R148H]

Thanks for the reply. Well, with a bit of fiddling, I managed to get 2 versions of FAH to run independently with different slots and config info [on windows]. More or less, it involves changing the config.xml file in each respective FAH repository to customise slots and user info, then running the FAHClient application. 2 windows appear and connect independently, but as you can only open one FAHControl at a time, it seems like whichever is launched first takes lock priority over the FAHControl viewer.

Coincidentally, I also found two pages relating to slots and customisation https://foldingathome.org/faqs/configur ... l-setting/ and https://foldingathome.org/faqs/configur ... l-setting/. I wonder if you can pass in other config info flags to these name-value pairs?

And I wonder how this process can be replicated on linux seeing as Linux tends to spread loads of core files all over the place, not just in /var/lib/...

[Whompithian]

I have three instances of FAHclient running on Centos 8 Stream. I have three different configs stored under /etc/fahclient, three separate directories under /var/lib/fahclient/, and three separate systemd units to launch them. Each instance of the client has a different command-port and http-addresses value set to avoid collisions (36330, 36331, and 36332 for command-port; 7396, 7397, and 7398 for http-addresses). "<gpu v='false'/>" is set in all config files to avoid unexpected changes. The correct config is then passed to systemd. One systemd ExecStart line looks this:

Code: Select all

ExecStart=/usr/bin/FAHClient --config=/etc/fahclient/gpu1.xml --chdir=/var/lib/fahclient/gpu1/

I don't use FAHControl, though. When I need to send a command, I use FAHClient directly so that I can pass the config that has the command port for the correct instance. I set up several shell aliases to make this easier. If FAHControl lets you specify arbitrary addresses to connect to, then it might allow you to specify non-default ports to use, as well. In that case, it should work fine with a setup similar to what I have running.

[PRP_R148H]

Thank you Whompithian! I think I've got it working (you gave me the confidence that it was possible). I had a little dig around my kubuntu setup to try and emulate your structure. Here's what I've done:

1) Went to `/etc/fahclient/` and made 2 config files (gpu1.conf and gpu2.conf) with relevant usernames, GPU slots, setting <gpu v='false'/> so the conf files don't get auto-updated etc
2) Went to `/var/lib` and duplicated the fahclient folder to fahclient2
3) Run each client using `sudo FAHClient --config=/etc/fahclient/gpu1.xml --chdir=/var/lib/fahclient` and `FAHClient --config=/etc/fahclient/gpu2.xml --chdir=/var/lib/fahclient2`

Both cards appear to be running separately in their individual clients; I'll check back tomorrow and confirm.

One thing I didn't change was the HTTP-addresses or command ports. I was expecting that to be my hurdle, but it seems to work fine without any changes yet?

[Neil-B]

OK ... I'm intrigued ... what is the use case for needing to run two clients on the same linux machine? ... I ask as with the current client I ddn't think this needed to be done any more for many of the old use cases ... Knowing why would allow devs to maybe resolve this in future releases.

[Joe_H]

Thanks for the reply. Well, with a bit of fiddling, I managed to get 2 versions of FAH to run independently with different slots and usernames/passwords [on windows]. More or less, it involves changing the config.xml file in each respective FAH repository to customise slots and user info, then running the FAHClient application. 2 windows appear and connect independently, but as you can only open one FAHControl at a time, it seems like whichever is launched first takes lock priority over the FAHControl viewer.

Coincidentally, I also found two pages relating to slots and customisation https://foldingathome.org/faqs/configur ... l-setting/ and https://foldingathome.org/faqs/configur ... l-setting/. I wonder if you can pass in username/team info to these name-value pairs?

And I wonder how this process can be replicated on linux seeing as Linux tends to spread loads of core files all over the place, not just in /var/lib/...

The way the options were passed years ago by the persons who reported doing it within a single client was adding Extra Slot Options under the Configure / Slots as mentioned in the first of the two links. That would be the parameters "user", "team", and "passkey" set to the values you want. If the team number is the same for both, the setting for the client as a whole will suffice. Same for the passkey, it can be used with more than one username.

As I mentioned, no one has tried this in a while and reported on this method working or not, so changes in the client over the last half dozen or so years may keep this from working now.

As for Linux, the client does not place folding cores all over the place. They go in a cores subdirectory based on the CWD:. In a default install that will be a subdirectory located in the same directory as the config.xml file, GPUS.txt file, and log.txt file.

When run from the command line the path to the working directory is either based on the current directory or a path needs to be provided.

The default install works the same on Windows, a set location is made for the data directory. The only difference is that the most recent versions of the client place the data directory in a location that is potentially accessible to all users, the older versions placed that directory base on the user logged in and installing the client.

[bruce]

I'm having a hard time figuring out any situation where running multiple clients is able to accomplish anything that you can't accomplish by carefully managing the options in config.xml.

What is the situation that makes you WANT to run multiple clients?

[Whompithian]

For me, the motivation is to decouple the slots. With a single client, if one slot gets a work unit that misbehaves terribly and needs to be dumped manually, then that requires interrupting the work being done on the other slots in order to release the lock on the work database so that I can request the misbehaving work unit be dumped. By running each slot under its own client, I can stop a single slot and dump a bad work unit while the other work units continue to run, greatly reducing the risk of destabilizing one of those work units and having it involuntarily dumped. It's extra work that I don't care to implement, but I've never been able to get the client to handle bad work units gracefully.

[JimboPalmer]

Oddly, I never get 'misbehaving' WUs, I suspect your setup.

Using the the Bonus Status on this page https://apps.foldingathome.org/

I have 120,972 Downloaded WUs, (in 12 years) and 189 of them Expired, for a 99.84% completion ratio. It is just not worth sweating over.

[bruce]

Oddly, I never get 'misbehaving' WUs

Same here. I almost never need to manually dump a WU and you shouldn't have to either. Show us the log of your situation and we'll help you fix whatever is misbehaving.

[Whompithian]

Show us the log of your situation and we'll help you fix whatever is misbehaving.

I'm going to decline this offer. You asked why someone would want to run multiple clients and that is my reason. If my GPUs get locked at 100% usage by FAHCore_22 and I see no signs of progress after a day, I'm going to dump the work unit, not wait for it to expire.

[Neil-B]

That is obviously your right ... I believe what was being offered was help sorting out why you are having issues that mean you feel the need to dump since these issues are impacting progress of wus and of the science ... you shouldn't need to resort to dumping wus

[bruce]

Show us the log of what conditions led to 100% utilization with with zero progress after a day. Something is seriously wrong and we want to prevent that from happening to others.

[Whompithian]

As do I, but I resorted to a full system restore of a two-month old image that seems to have stabilized things. Unfortunately, I did not export the logs before overwriting the partition with the backup image. I only saved the contents of /etc. If I encounter similar behavior in the future, I will be sure to open a thread and post the log.