Configurable affinity changer for Windows

Xilikon · Post by **Xilikon** » Tue Sep 16, 2008 10:22 pm

A simple comment... I just looked at some of your source files and I will applaud for doing a very clean code (well indented and grouped). It's a hell of a long time since the last time I coded in C (close to 10 years) but it's very clear. If I find a bug, I will try to hunt it in the code and tell you what to fix since I don't have VS.NET installed.

baobab · Post by **baobab** » Wed Sep 17, 2008 4:36 am

Thanks. I have fair bit of experience with C/C++, but on Unix systems. This is my first Windows application since Win 3.1 days.

@l8nights: you can set the log level to any positive value, but the lowest priority messages are at level 5 (6 in v0.81), so you will not get anything more verbose above that.

baobab · Post by **baobab** » Fri Sep 19, 2008 6:53 pm

Version 0.81 of AffinityChanger has been released and it is available from the same place: http://sourceforge.net/project/platform ... _id=238550

New features:
- added support for custom priority settings. Priority is one of the optional profile attributes.
- added new log messages that display a warning if there are no valid application profiles, or if an application profile does not match any running process when program starts. Requires LogLevel 4 or higher.

New file releases:
- added release for 64 bit versions of Windows

Bug fixes:
- fixed a rare race condition that could have caused processes to be missed and later ignored if the target process was loading at the same time AffinityChanger was checking for new processes.
- cleaned up the documentation a bit.

The most up-to-date documentation is included with the program. Currently I am locked from modifying the online documentation while SourceForge.net are migrating some of their servers. I will update the online documentation when it will be possible to do so.

ElectricVehicle · Post by **ElectricVehicle** » Fri Sep 26, 2008 4:37 am

Odd, CPU detection failed.

Here's the error message. More in code block below.

Error build_cpu_sets: found sets of different sizes, 1-2.

Windows Vista 32 bit, Core 2 Quad Q6600 (Speed Step enabled, C1E enabled, OC'd to 2.6 GHz (290 x 9) )
Run CMD window as administrator.
cd to directory with AffinityChanger
run AffinityChanger

Fails with these errors about 3 out of 4 times:
CPU may have been down at 6x instead of 9x mutiplier since only GPU client was running to start and it's not enough CPU load for the CPU to go fast.

Code: Select all

C:\Users\GatoPrimo\Bin\AffinityChanger>AffinityChangerConsole.exe
AffinityChanger 0.81, (c) 2008 <[email protected]>

Auto detecting CPU configuration. Please wait.
|==================================================| 100.0%
Error build_cpu_sets: found sets of different sizes, 1-2.
Error build_cpu_sets: Please send the above error message and the numbers listed
 below to: [email protected]
1, 2, 1276.25
0, 2, 1325.31
1, 3, 1369.15
0, 3, 1372.16
0, 1, 8468.29
2, 3, 12985.4
Error build_cpu_sets: Unable to determine the logical CPU sets. Use only physica
l CPU IDs.
Found 4 CPUs:
         CPU0 has mask 00000001
         CPU1 has mask 00000002
         CPU2 has mask 00000004
         CPU3 has mask 00000008
Warning process_nested_cpus::operator (): no logical CPU set of type PAIR was de
tected on this machine.
Warning parse_input_file: Ignoring profile line: C:\Users\GatoPrimo\FAH\FahCore_
*.exe := PAIR0 [assign=1,resource=CPUUSE,policy=PSEUDOBALANCED]
Warning process_nested_cpus::operator (): no logical CPU set of type PAIR was de
tected on this machine.
Warning parse_input_file: Ignoring profile line: C:\Users\GatoPrimo\Fah2ndSmpCli
ent\FahCore_*.exe := PAIR1 [assign=1,resource=CPUUSE,policy=PSEUDOBALANCED]
ONE TIME CHECK: There is no valid application profile. Press Ctrl+C to terminate
 the program at any time.
^C

The input file:

Code: Select all

#
# Affinity Changer Console Input File
#
# Lines that start with a '#' and empty lines are ignored. 
# Configuration lines include global flag setting lines, and application profile
# lines. These lines can be interleaved in any order. However, the order in which
# you specify the various application profiles matters if a process could be
# matched by multiple application profiles.

# To run AffinityChanger in TestMode (no affinity settings are modified),
# uncomment the line below and specify a value different than zero. To disable 
# test mode, set the value to 0 or comment the line below.
# It is recommended that you try the input file in test mode first, after you
# make changes, especially if you are unfamiliar with the syntax.
TestMode = 1

# There are several flags that control the verbosity of the program. In general,
# a higher value means a more verbose output. 

# By default the warning level is set to 2. To change it, uncomment the next 
# line and specify a value>=0. Zero disables warning messages (not recommended).
#WarningLevel = 2

# Logging is done on both console and in a file called 'affinitylog.txt'. 
# Separate flags control the verbosity for each medium. By default the logging
# level is set to 3. Uncomment the next lines to change the values of these 
# flags.
# To change only the console verbosity:
#ConsoleLogLevel = 3
#
# To change only the file verbosity:
#FileLogLevel = 3
#
# To change both mediums at the same time:
LogLevel = 3

# One can change the value of a flag multiple times. In these cases the 
# most recently specified value will take effect.
#

##################################################
##          Application Profile lines
##################################################
#
# An application profile is specified on a single line. 
# An application profile includes the following information: an application
# name, a CPU mask, and optional attributes in the following format:
# C:\Path\To\Application := CPU0+CPU1 [attr1=val1,attr2=val2]
#
# Check the documentation for more information about these fields.

# While AffinityChanger does not target any particular application, its
# initial development was motivated by the Folding@Home project. Therefore,
# I will provide several Application Profile examples inspired by the FAH
# project.

# This example assume a machine with an Intel Quad Core CPU for which 
# AffinityChanger detected the two PAIRs of cores that share the L2 cache. 
# All profiles use logical CPU IDs in the CPU mask.

# User runs two Windows SMP clients. We assign each client to two cores 
# that share an L2 cache. The two SMP clients are installed in different 
# directories. We use this information to distinguish between their processes:
C:\Users\GatoPrimo\FAH\FahCore_*.exe := PAIR0 [assign=1,resource=CPUUSE,policy=PSEUDOBALANCED]
C:\Users\GatoPrimo\Fah2ndSmpClient\FahCore_*.exe := PAIR1 [assign=1,resource=CPUUSE,policy=PSEUDOBALANCED]

#
# You should try any scenario under the TestMode first to verify that it
# does what you expect.
#

I'm working fine at the moment, I just terminated the program and kept re-running it until it started successfully. Nice little program, thanks for developing it!

Xilikon · Post by **Xilikon** » Fri Sep 26, 2008 1:07 pm

Interesting bug, I guess when C1E kicked, it lowered the multiplier and it messed up the detection or due to varying load, the multiplier is moving and it found inconsistencies.

Try to apply a load on the cpu by starting the F@H client then restart the AC program to see if the issue is gone.

ElectricVehicle · Post by **ElectricVehicle** » Fri Sep 26, 2008 11:49 pm

I hear C1E may be evil for overclocking, so I may disable it anyway. I like the idea of the energy savings, but if my 24x7 folder is idling - IT"S A MISTAKE!

Hmmm.. well my machine is right here and I don't have to trash a WU to try...

^C stopped it and restarted it 5 times in a row without a single error. Seems like it won't happen with a loaded CPU. The load is two instances of the SMP client and a GPU2 Nvidia client all running concurrently.

So the working theory is that changing CPU performance during it's detection phase may cause the detection to fail.

When I tried it before I believe I did it with nothing significant running and then a few times with the Nvidia GPU2 client running on Vista but that client doesn't use much CPU, so the CPU stays at the low 6x multiplier, even with the GPU2 client running.

Ps. I've only managed to OC my 2 Q6600 Stepping G0 CPUs in separate computers to 2.6 and 2.8 GHz even at 1.33 V Vcore, which seems pretty low from what I've seen from others. The stock VID for the 2 particular Q6600s I got is 1.200 V. From other posts that would suggest they might be reasonable or good over clockers. Could the C1E be the likely cause of my poor overclocking results so far?

(Asus P5B Deluxe motherboard, Corsair PC6400 DDR2 ram, so all the other components should support good overclocks. Core temps below 48 C.)

baobab · Post by **baobab** » Sat Sep 27, 2008 3:35 am

ElectricVehicle,thanks for the report.
Yes, the error is due to cores 0 and 1 being in the C1E state when the program tested that link, while during the test for cores 2 and 3 the chip was at full speed.

But this is a problem that I would like to fix. In fact the program already tries to put some load on the CPU before it does the detection to avoid exactly this problem that you are seeing, but I guess the load was too short for your machine.
I tried to reproduce the problem on my machine with no load, or just with the GPU client running on one core, but it always measured the chip at full speed.
I will experiment with this a bit more.

I am planing to release another bug fix version by the middle of next week which hopefully will eliminate or at least reduce the occurrence of this bug, and also to address a problem that Xilikon reported.

baobab

ElectricVehicle · Post by **ElectricVehicle** » Sun Sep 28, 2008 9:06 am

First, let me say once again, thank you for Affinity Changer!

AffinityChanger can't find my fah processes...

Initially it would start and not generate any messages after processor detection.

So I set the messaging to:
LogLevel = 99
WarningLevel = 99

Then I got these warnings (see code block for many more of the same):
Warning Create snapshot of modules: 5 (Access is denied)

I tried running AffinityChanger using "Run this program as Administrator" in the Properties dialog, Compatibility tab. No matter what I did, it could not see my FahCore_a1.exe processes. I put in a wildcard entry for everything in the client directory and you can see it matched some fah processes but not the cores.

I run fah under my main login that has Administrative privileges, miktadmin. The fah cores (FahCore_a1.exe) are running as worker processes under the user name fah, which does not have Administrative privileges.

So it appears the problem is that even with Administrative privileges AffinityChanger is not able to get the process information (Windows error: 5 (Access is denied)) for the processes running under another user id. Under default message settings there is no indication that AffinityChanger is ignoring processses that match the profile due to Access is denied errors.

This is under Windows Vista 32 bit. (UAC (User Access Control) is at the default setting of enabled)
I didn't experience this issue on my other computer since I have fah and the fah core processes all running under the same user name that has Administrative privileges.

Code: Select all

C:\Users\miktadmin\Documents\bin>AffinityChangerConsole
AffinityChanger 0.81, (c) 2008 <[email protected]>

Auto detecting CPU configuration. Please wait.
|==================================================| 100.0%
Found 4 CPUs:
         CPU0 has mask 00000001
         CPU1 has mask 00000002
         CPU2 has mask 00000004
         CPU3 has mask 00000008
Found 2 PAIRs:
         PAIR0<CPU0,CPU1> has mask 00000003
         PAIR1<CPU2,CPU3> has mask 0000000C
9/27/2008 23:38:04 PM: Live check Warning Create snapshot of modules: 5 (Access
is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Warning Create snapshot of modules: 5 (Access is denied)
Setting process affinity using Pseudo Balanced Policy:
 - process 5168 with name C:\Fah\mpiexec.exe uses   0.0% of CPU time
 C:\Fah\mpiexec.exe with PID 5168 using  0.0% of CPU time, set affinity to CPU0
 - process 1064 with name C:\Fah\[email protected] uses   0.0% of CPU t
ime
 C:\Fah\[email protected] with PID 1064 using  0.0% of CPU time, set af
finity to CPU1
Setting process affinity using Pseudo Balanced Policy:
 - process 5644 with name C:\FahSmp2ndClient\[email protected] uses   0
.0% of CPU time
 C:\FahSmp2ndClient\[email protected] with PID 5644 using  0.0% of CPU
time, set affinity to CPU2
 - process 3980 with name C:\FahSmp2ndClient\mpiexec.exe uses   0.0% of CPU time

 C:\FahSmp2ndClient\mpiexec.exe with PID 3980 using  0.0% of CPU time, set affin
ity to CPU3
ONE TIME CHECK: No processes found for profile "C:\FAH\FahCore_*.exe"
Setting process affinity using Pseudo Balanced Policy:
ONE TIME CHECK: No processes found for profile "C:\Fah\FahCore_*.exe"
Setting process affinity using Pseudo Balanced Policy:
ONE TIME CHECK: No processes found for profile "C:\FahSmp2ndClient\FahCore_*.exe
"

baobab · Post by **baobab** » Sun Sep 28, 2008 9:47 pm

ElectricVehicle wrote: AffinityChanger can't find my fah processes...

Thanks for the report again. This is a known limitation.
It is not that AffinityChanger cannot find your processes. It does not have enough privileges to obtain the full name of processes running under a different user account, so it cannot even determine that they match any of your application profiles. It also would not have privileges to actually change affinity for those processes. Can you manually change the affinity of processes running under user name "fah" in Windows Task Manager? For me it does not work in XP to change affinity of processes running under a different user account, even though I use an account with Administrator privileges.

Anyway, after some more reading through MSDN I found a way to get around this restriction. It works for me in XP, and now I can change affinities even for System processes.

Would you be willing to beta test this version on your Vista machine before I release it? I need to confirm that it works in Vista as well.
If you agree, I will PM you with a way to get the program.

ElectricVehicle · Post by **ElectricVehicle** » Mon Sep 29, 2008 2:56 am

I'd love to test it! Just let me know how - here, via private message, whatever...

Yes, I changed the affinities manually in Vista Task Manager after selecting "Show processes from all users". When you check that option you get the User Account Control dialog asking you for permission.

Depending on your fix, if there's some mode where AffinityChanger might be running and can't change those processes, it might do a one time warning at start up stating that "Warning: Process <x> running as User Name <User Name> matches profile but Affinity Changer has insufficient privileges to change it's affinity." Where <x> is the nameof the process, say C:\FAH\FahCore_a1.exe

When this gets resolved in whatever manner, if you can add a short note to the documentation about it. I think I read it thoroughly and didn't see mention that Affinity Changer couldn't change affinity for processes running as users besides the one Affinity Changer is running as.

Ps. for grins, I tried running Affinity Changer in the XP Compatibility mode when I first ran into the issue, thinking that XP's looser security model might get away with it. But as you pointed out, that doesn't work in the XP envrionment.

By the way, are you aware of Sysinternals? The people doing it have been doing really cool process, file and other close to the operating system utilities along with some explanations. You can really tell what's going on in your system using the Sysinternals tools.

One new tool they just did you may find interesting, at least for testing or validation:
Coreinfo 1.0
Coreinfo is a new command-line utility that shows you the mapping between logical processors and the physical processor, NUMA node, and socket on which they reside, as well as the cache’s assigned to each logical processor.

Affinity Changer on my two computers is going to bump my contribution to FAH by another 1000+ PPD! Valuable little program!

ElectricVehicle · Post by **ElectricVehicle** » Mon Sep 29, 2008 5:33 am

Hehe, stay tuned, baobab is a busy boy improving his tool for all of us!

baobab · Post by **baobab** » Wed Oct 01, 2008 6:54 am

Version 0.82 of AffinityChanger has been released and it is available at the usual place: http://sourceforge.net/project/platform ... _id=238550
The most recent documentation can be found online at http://affinitychanger.sourceforge.net and it is also packaged with the program.

Thanks for Xilikon, ElectricVehicle and l8nights for bug reports and suggestions. This version addresses mainly their suggestions and bug reports.

New features:
- user can modify the default time interval between affinity readjustments through a configuration file flag called TimeInterval;
- added a new optional profile attribute "force" which causes AffinityChanger to re-set affinities for matching processes if they are changed by another program or manually by the user. Previous versions and the default behavior in version 0.82 is to change affinities for processes only first time they are encountered or if a readjustment is needed because of a change in the number of matching processes or their load;
- new command line parameter "-startdelay" to specify a delay in seconds before the program starts executing. This flag is useful if AffinityChanger is set to run automatically when Windows starts up, to allow Windows to finish initialization a bit faster;
- ability to change affinities and priorities for processes running under different user accounts and for most system processes if AffinityChanger is started as Administrator.

Bug fixes:
- improved the method of waking CPUs from a low power state to perform the CPU detection heuristic. This method should reduce the number of failed detections;
- other minor changes.

Anachron · Post by **Anachron** » Sat Oct 11, 2008 12:04 pm

The heuristics does not seem to work here, as i get this every time:

Code: Select all

E:\Programmer\AffinityChanger-0.82>AffinityChangerConsole.exe
AffinityChanger 0.82, (c) 2008 <[email protected]>

Auto detecting CPU configuration. Please wait.
|==================================================| 100.0%
Error build_cpu_sets: found sets of different sizes, 3-4.
Error build_cpu_sets: Please send the above error message and the numbers listed
 below to:[email-address removed]
1, 2, 1429.22
2, 3, 1487.94
0, 2, 1523.13
1, 3, 1559.13
0, 1, 1622.58
0, 3, 1632.42
Error build_cpu_sets: Unable to determine the logical CPU sets. Use only physica
l CPU IDs.
Found 4 CPUs:
         CPU0 has mask 00000001
         CPU1 has mask 00000002
         CPU2 has mask 00000004
         CPU3 has mask 00000008
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@home-gpu\[email protected]
e with PID 1468, set affinity to CPU0
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@home-gpu\FahCore_11.exe
with PID 2288, set affinity to CPU0
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@Home-smp\FahCore_a1.exe
with PID 3612 using 138492.0KB of RAM, set affinity to CPU1
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@Home-smp\FahCore_a1.exe
with PID 3608 using 97796.0KB of RAM, set affinity to CPU2
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@Home-smp\FahCore_a1.exe
with PID 2500 using 96248.0KB of RAM, set affinity to CPU3
11.10.2008 13:51:28: E:\Programmer\Folding@home\Folding@Home-smp\FahCore_a1.exe
with PID 3592 using 95236.0KB of RAM, set affinity to CPU3

I have a Quad Phenom 9850.

The program works fine except for that error.

Is it possible to not show the console window? i would like to just run it as a background process.

Also, I want to assign SMP folding to three of the cores, as the GPU eats the first core. Is it best to assign two of the fahcores to the same cpu-core, or to have the last fahcore assigned to the three cpu-cores used for SMP?
If the latter, is it possible to do this with AffinityChanger?

Xilikon · Post by **Xilikon** » Sat Oct 11, 2008 11:38 pm

Phenoms have 4 cores in a single die so there is technically no cpu pair to be determined. Even if the heuristic detection fails, it will still work since any pair you pick will be ok. However, the program could be improved by first determining what cpu type and to pick a algorithm depending on which one (if it is a Intel quad, determine cpu pairs but if it is a Phenom, just form virtual pairs).

baobab · Post by **baobab** » Sat Oct 11, 2008 11:54 pm

Hi Anachron,

Thanks for the report. For the AMD quad cores you get nothing from the CPU heuristic because all cores are symmetric.
For now you could run it with the command line parameter "-nodetect" and use only physical CPU IDs (CPU0, CPU1, etc) in your configuration file.
However, I will adjust the thresholds in the heuristic so it will detect a QUAD logical cluster, as it should. I had no AMD quad core machine to test the heuristic before, so I appreciate that you reported your results. This is a minor issue though because as I said there are no asymmetries with your machine. So I will not release a separate version just for this fix. The change will be in the next version of the program.

Which leads me to your second point. I plan to make a system tray variant of the program and this is what I am going to work on next. But this can take from one week to three weeks. I can work on it only when I have free time and I do not have a clear idea what it involves at this point. This is my first Windows program.

Regarding your last question, I do not know which variant is optimal. You can either assign all four processes to three of the cores with one core reserved for the GPU client:

Code: Select all

C:\PathToGPUClient\FahCore_*.exe := CPU0
C:\PathToWindowsSMPClient\FahCore_*.exe := CPU1 + CPU2 + CPU3

or assign one process per core, except for one core which gets assigned the two lightest processes (GPU rule is the same, I will write only the rule for the SMP client):

Code: Select all

C:\PathToWindowsSMPClient\FahCore_*.exe := CPU1+CPU2+CPU3 [assign=1,resource=CPUUSE,policy=PSEUDOBALANCED]

In this scenario, CPU3 will get assigned the two lightest processes.

And you can also specify the policy where each core gets assigned one process, but the lightest process gets assigned to all three cores. This one is a bit more involved:

Code: Select all

C:\PathToWindowsSMPClient\FahCore_*.exe := CPU1+CPU1+CPU1+CPU2+CPU2+CPU2+CPU3+CPU3+CPU3+CPU1+CPU2+CPU3 [assign=3,resource=CPUUSE,policy=PSEUDOBALANCED]

Make sure you write the rule on a single line. There should be no newline in the middle of a rule.
Here I used a trick. I assign 3 CPUs to each process. But the first three processes will get assigned only one CPU because I specified the same CPU three times. At the end I specified all three CPUs, so the last process will get assigned to all three cores. Hope it makes sense to you.
You have to test to see which variant is optimal. This last one might be.

Cheers,
baobab

Edit: Xilikon, the program does not need to know the architecture. It tries to detect clusters of cores/CPUs that have an affinity for each other. For an AMD quad, the four cores have an equal affinity for each other. It will not detect PAIRs of cores, but a QUAD as it is explained in documentation: http://affinitychanger.sourceforge.net/#cpudetection
This makes more sense for dual socket systems where it should detect two QUADs. I do not know if it will work correctly until somebody tests it though.

Folding Forum

Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows

Re: Configurable affinity changer for Windows