Change in BA requirements

[troy8d]

It appears that most everybody can agree that the decrease in points of 50% to 75% when switching from bigadv to smp is both disheartening and unpalatable to those affected by it. Further, it appears this far exceeds the design parameters of the bigadv program Dr. Kasson outlines, more than three times the intended 16.7% decline in points when switching from bigadv to smp WUs.

My initial hunch was that, due to the exponentially decreasing QRB, the large decrease in points was a failure of points system to properly account for the speed of a bigadv machine folding smp WUs much faster than bigadv WUs. A few quick back of the envelope calculations show that a bigadv machine getting 200,000 PPD on a smp WU is completing WUs 4.7 times faster than an "average" smp machine that gets 20,000 PPD on the same WU is only receives a QRB that is 2.2 times higher. The comparison between a 10,000 PPD and a 20,000 PPD machine is quite different. The faster machine is completing WUs 1.6 times faster and receives a QRB multiplier that is 1.3 times greater. It is readily apparent that additional speed increases on slow machines are rewarded much more than equal increases on fast machines. Return speed of WUs is initially highly valued, but adding additional return speed is subject to exponentially diminishing marginal returns..

Comparing bigadv vs smp on the same machine, however, provides an interesting result. The extremely large k-factor for bigadv WUs creates a roughly equal QRB multiplier for bigadv and smp WUs on the same machine across a wide range of WUs. If the k-factor is intentionally calculated to achieve this result (and consistency across a range of WUs leads me to believe it is intentional), the reason for the large decline in points when transitioning from bigadv to smp would not lie in the QRB. This leaves the assigned point as the most probable cause for the large deviation for the anticipated 17% decline in points. This is further supported by what Dr. Kasson indicates above, there are scaling issues both ways on bigadv vs smp (bigadv WUs fold much faster on a bigadv machine relative to the benchmark machine while the opposite is true for smp).

A potentially novel solution to the benchmark scaling issue may be to simply change the benchmark machine to a bigadv class machine. This would minimize the scaling issues (both up and down) for big advantage machines and provide stability through the bigadv to smp WU transition. Additionally, as a result of this change we can expect a bit of a point boost on non-bigadv machines running smp WUs that do not scale well to a large number of cores. Most people have argued that these lower smp WUs are under-valued and under-appreciated in regard to the points they receive, potentially making the point boost an ancillary benefit rather than a detriment. Regardless, at a minimum it would simplify the scaling issues by reducing them to one direction.

If this does not resolve the benchmarking issues, I fear the solution becomes a bit more complicated. The long-term solution may require that benchmarking procedures be changed and the golden standard of a single benchmark machine for all WUs be abandoned as the real world results indicate that it is not rewarding "equal pay for equal work." (I use the term work as a measurement of science). For bigadv class machines, the aforementioned scaling issues and observed data points suggest that, on a bigadv machine the bigadv points are over-inflated and the smp points are under-inflated relative to the theoretical construct of the points system. It would be extremely disheartening if PG chose to sacrifice the integrity of the observed results of the points system to maintain its underlying theoretical integrity.

In attempting to break away from the strict adherence to "equal pay for equal work" it is important to note that it appears bigadv was never based purely upon the premise. It is clear both from Dr. Kasson's description of bigadv and the observed comparison between bigadv and smp that the bigadv points premium rewards factors other than pure work/science, negating the direct application of equal pay for equal work to big adv WUs.

Given that actual results of bigadv vs smp PPD manifest large deviations from the intended behavior, the aforementioned difficulty in benchmarking bigadv WUs using current procedures and that equal pay for equal work only loosely applies to bigadv WUs; it may be time to consider revising benchmarking procedures to produce more consistent results. Several years ago, GPU WUs were benchmarked separately from smp WUs because it was not a valid comparison (and infeasible at the time). The description of how bigadv WUs are benchmarked makes it very much appear as if the integrity of the points system is being compromised for the sake of a maintaining its theoretical purity.

Other potential approaches to addressing the large decline in points transitioning from bigadv to smp can be achieved by modifying the points system. If benchmarking is at the heart of the problem, however, this approach will be a second-best solution. One possible approach is to alter the points formula to punish increased return speed less harshly than the square root function does (outlined above). I've briefly played around with a few functional forms that do so without drastically altering the points landscape. This approach would, however, reward increased points to all smp folders as well as QRB GPU folders. An alternate approach would be to create a smalladv WU type with a more appropriate decline in points relative to the bigadv level. Smalladv could be similar to bigadv but targeted at some of the larger SMP WUs that scale well with more cores and would benefit from the increased folding horsepower that no longer qualifies for bigadv.

Regardless of the cause, it is readily apparent that the bigadv point system is not functioning within the design parameters Dr. Kasson described, and this has understandably upset a large number of donors that will be negatively affected by the upcoming changes. Any changes to benchmarking procedures or points system will have to be carefully considered and probably will not be in place by the announced deadline for upcoming bigadv changes. As a show of good faith, it would be a nice gesture if PG delayed the impending changes until a solution is worked out or a temporary stop-gap measure can be put in place. This would prevent affected donors from experiencing a massive decline in points that it appears they were never intended to face if bigadv points functioned as originally outlined. While it is important that the folding community embraces change and continues to move forward, it is equally important to fix the issues at hand before doing so. The biennial cycle of bigadv crises has quickly grown wearisome.

[kerryd]

It appears that most everybody can agree that the decrease in points of 50% to 75% when switching from bigadv to smp is both disheartening and unpalatable to those affected by it. Further, it appears this far exceeds the design parameters of the bigadv program Dr. Kasson outlines, more than three times the intended 16.7% decline in points when switching from bigadv to smp WUs.

My initial hunch was that, due to the exponentially decreasing QRB, the large decrease in points was a failure of points system to properly account for the speed of a bigadv machine folding smp WUs much faster than bigadv WUs. A few quick back of the envelope calculations show that a bigadv machine getting 200,000 PPD on a smp WU is completing WUs 4.7 times faster than an "average" smp machine that gets 20,000 PPD on the same WU is only receives a QRB that is 2.2 times higher. The comparison between a 10,000 PPD and a 20,000 PPD machine is quite different. The faster machine is completing WUs 1.6 times faster and receives a QRB multiplier that is 1.3 times greater. It is readily apparent that additional speed increases on slow machines are rewarded much more than equal increases on fast machines. Return speed of WUs is initially highly valued, but adding additional return speed is subject to exponentially diminishing marginal returns..

Comparing bigadv vs smp on the same machine, however, provides an interesting result. The extremely large k-factor for bigadv WUs creates a roughly equal QRB multiplier for bigadv and smp WUs on the same machine across a wide range of WUs. If the k-factor is intentionally calculated to achieve this result (and consistency across a range of WUs leads me to believe it is intentional), the reason for the large decline in points when transitioning from bigadv to smp would not lie in the QRB. This leaves the assigned point as the most probable cause for the large deviation for the anticipated 17% decline in points. This is further supported by what Dr. Kasson indicates above, there are scaling issues both ways on bigadv vs smp (bigadv WUs fold much faster on a bigadv machine relative to the benchmark machine while the opposite is true for smp).

A potentially novel solution to the benchmark scaling issue may be to simply change the benchmark machine to a bigadv class machine. This would minimize the scaling issues (both up and down) for big advantage machines and provide stability through the bigadv to smp WU transition. Additionally, as a result of this change we can expect a bit of a point boost on non-bigadv machines running smp WUs that do not scale well to a large number of cores. Most people have argued that these lower smp WUs are under-valued and under-appreciated in regard to the points they receive, potentially making the point boost an ancillary benefit rather than a detriment. Regardless, at a minimum it would simplify the scaling issues by reducing them to one direction.

If this does not resolve the benchmarking issues, I fear the solution becomes a bit more complicated. The long-term solution may require that benchmarking procedures be changed and the golden standard of a single benchmark machine for all WUs be abandoned as the real world results indicate that it is not rewarding "equal pay for equal work." (I use the term work as a measurement of science). For bigadv class machines, the aforementioned scaling issues and observed data points suggest that, on a bigadv machine the bigadv points are over-inflated and the smp points are under-inflated relative to the theoretical construct of the points system. It would be extremely disheartening if PG chose to sacrifice the integrity of the observed results of the points system to maintain its underlying theoretical integrity.

In attempting to break away from the strict adherence to "equal pay for equal work" it is important to note that it appears bigadv was never based purely upon the premise. It is clear both from Dr. Kasson's description of bigadv and the observed comparison between bigadv and smp that the bigadv points premium rewards factors other than pure work/science, negating the direct application of equal pay for equal work to big adv WUs.

Given that actual results of bigadv vs smp PPD manifest large deviations from the intended behavior, the aforementioned difficulty in benchmarking bigadv WUs using current procedures and that equal pay for equal work only loosely applies to bigadv WUs; it may be time to consider revising benchmarking procedures to produce more consistent results. Several years ago, GPU WUs were benchmarked separately from smp WUs because it was not a valid comparison (and infeasible at the time). The description of how bigadv WUs are benchmarked makes it very much appear as if the integrity of the points system is being compromised for the sake of a maintaining its theoretical purity.

Other potential approaches to addressing the large decline in points transitioning from bigadv to smp can be achieved by modifying the points system. If benchmarking is at the heart of the problem, however, this approach will be a second-best solution. One possible approach is to alter the points formula to punish increased return speed less harshly than the square root function does (outlined above). I've briefly played around with a few functional forms that do so without drastically altering the points landscape. This approach would, however, reward increased points to all smp folders as well as QRB GPU folders. An alternate approach would be to create a smalladv WU type with a more appropriate decline in points relative to the bigadv level. Smalladv could be similar to bigadv but targeted at some of the larger SMP WUs that scale well with more cores and would benefit from the increased folding horsepower that no longer qualifies for bigadv.

Regardless of the cause, it is readily apparent that the bigadv point system is not functioning within the design parameters Dr. Kasson described, and this has understandably upset a large number of donors that will be negatively affected by the upcoming changes. Any changes to benchmarking procedures or points system will have to be carefully considered and probably will not be in place by the announced deadline for upcoming bigadv changes. As a show of good faith, it would be a nice gesture if PG delayed the impending changes until a solution is worked out or a temporary stop-gap measure can be put in place. This would prevent affected donors from experiencing a massive decline in points that it appears they were never intended to face if bigadv points functioned as originally outlined. While it is important that the folding community embraces change and continues to move forward, it is equally important to fix the issues at hand before doing so. The biennial cycle of bigadv crises has quickly grown wearisome.

AMAN

[Nathan_P]

It appears that most everybody can agree that the decrease in points of 50% to 75% when switching from bigadv to smp is both disheartening and unpalatable to those affected by it. Further, it appears this far exceeds the design parameters of the bigadv program Dr. Kasson outlines, more than three times the intended 16.7% decline in points when switching from bigadv to smp WUs.

My initial hunch was that, due to the exponentially decreasing QRB, the large decrease in points was a failure of points system to properly account for the speed of a bigadv machine folding smp WUs much faster than bigadv WUs. A few quick back of the envelope calculations show that a bigadv machine getting 200,000 PPD on a smp WU is completing WUs 4.7 times faster than an "average" smp machine that gets 20,000 PPD on the same WU is only receives a QRB that is 2.2 times higher. The comparison between a 10,000 PPD and a 20,000 PPD machine is quite different. The faster machine is completing WUs 1.6 times faster and receives a QRB multiplier that is 1.3 times greater. It is readily apparent that additional speed increases on slow machines are rewarded much more than equal increases on fast machines. Return speed of WUs is initially highly valued, but adding additional return speed is subject to exponentially diminishing marginal returns..

Comparing bigadv vs smp on the same machine, however, provides an interesting result. The extremely large k-factor for bigadv WUs creates a roughly equal QRB multiplier for bigadv and smp WUs on the same machine across a wide range of WUs. If the k-factor is intentionally calculated to achieve this result (and consistency across a range of WUs leads me to believe it is intentional), the reason for the large decline in points when transitioning from bigadv to smp would not lie in the QRB. This leaves the assigned point as the most probable cause for the large deviation for the anticipated 17% decline in points. This is further supported by what Dr. Kasson indicates above, there are scaling issues both ways on bigadv vs smp (bigadv WUs fold much faster on a bigadv machine relative to the benchmark machine while the opposite is true for smp).

A potentially novel solution to the benchmark scaling issue may be to simply change the benchmark machine to a bigadv class machine. This would minimize the scaling issues (both up and down) for big advantage machines and provide stability through the bigadv to smp WU transition. Additionally, as a result of this change we can expect a bit of a point boost on non-bigadv machines running smp WUs that do not scale well to a large number of cores. Most people have argued that these lower smp WUs are under-valued and under-appreciated in regard to the points they receive, potentially making the point boost an ancillary benefit rather than a detriment. Regardless, at a minimum it would simplify the scaling issues by reducing them to one direction.

If this does not resolve the benchmarking issues, I fear the solution becomes a bit more complicated. The long-term solution may require that benchmarking procedures be changed and the golden standard of a single benchmark machine for all WUs be abandoned as the real world results indicate that it is not rewarding "equal pay for equal work." (I use the term work as a measurement of science). For bigadv class machines, the aforementioned scaling issues and observed data points suggest that, on a bigadv machine the bigadv points are over-inflated and the smp points are under-inflated relative to the theoretical construct of the points system. It would be extremely disheartening if PG chose to sacrifice the integrity of the observed results of the points system to maintain its underlying theoretical integrity.

In attempting to break away from the strict adherence to "equal pay for equal work" it is important to note that it appears bigadv was never based purely upon the premise. It is clear both from Dr. Kasson's description of bigadv and the observed comparison between bigadv and smp that the bigadv points premium rewards factors other than pure work/science, negating the direct application of equal pay for equal work to big adv WUs.

Given that actual results of bigadv vs smp PPD manifest large deviations from the intended behavior, the aforementioned difficulty in benchmarking bigadv WUs using current procedures and that equal pay for equal work only loosely applies to bigadv WUs; it may be time to consider revising benchmarking procedures to produce more consistent results. Several years ago, GPU WUs were benchmarked separately from smp WUs because it was not a valid comparison (and infeasible at the time). The description of how bigadv WUs are benchmarked makes it very much appear as if the integrity of the points system is being compromised for the sake of a maintaining its theoretical purity.

Other potential approaches to addressing the large decline in points transitioning from bigadv to smp can be achieved by modifying the points system. If benchmarking is at the heart of the problem, however, this approach will be a second-best solution. One possible approach is to alter the points formula to punish increased return speed less harshly than the square root function does (outlined above). I've briefly played around with a few functional forms that do so without drastically altering the points landscape. This approach would, however, reward increased points to all smp folders as well as QRB GPU folders. An alternate approach would be to create a smalladv WU type with a more appropriate decline in points relative to the bigadv level. Smalladv could be similar to bigadv but targeted at some of the larger SMP WUs that scale well with more cores and would benefit from the increased folding horsepower that no longer qualifies for bigadv.

Regardless of the cause, it is readily apparent that the bigadv point system is not functioning within the design parameters Dr. Kasson described, and this has understandably upset a large number of donors that will be negatively affected by the upcoming changes. Any changes to benchmarking procedures or points system will have to be carefully considered and probably will not be in place by the announced deadline for upcoming bigadv changes. As a show of good faith, it would be a nice gesture if PG delayed the impending changes until a solution is worked out or a temporary stop-gap measure can be put in place. This would prevent affected donors from experiencing a massive decline in points that it appears they were never intended to face if bigadv points functioned as originally outlined. While it is important that the folding community embraces change and continues to move forward, it is equally important to fix the issues at hand before doing so. The biennial cycle of bigadv crises has quickly grown wearisome.

Damn, that nicely sums up this 28 page thread.

[texinga]

Well said Troy. I hope that the leadership of PG can take what you have said to find some new and positive ways to move forward. I'm no mathematician, but you words and examples rang-true for me (having Folded Bigadv and SMP for several years). I think information like yours and your objectivity can be very helpful to the PG team.

Well done my friend!

[kasson]

Thank you for the suggestions. I need to consult with the rest of the F@H leadership, but I think the likelihood is that we'll want to move sooner rather than later.

I should also point out that one could equally conclude that BA work units are overvalued rather than that SMP are undervalued. A number of people have made this point, and it is not clear (and not entirely in my hands) which direction the points scheme will move. I would personally be in favor of a larger realignment, but that is of course complicated.

Historically, when we first introduced SMP work units, we used a Mac Pro as the benchmark machine. At the time, that machine was relatively powerful, and there was donor concern that a "high-end" machine was not the best choice of benchmark. It is good to remember history, but perhaps we have been over-mindful of it.

[orion]

I should also point out that one could equally conclude that BA work units are overvalued rather than that SMP are undervalued. A number of people have made this point, and it is not clear (and not entirely in my hands) which direction the points scheme will move. I would personally be in favor of a larger realignment, but that is of course complicated.

Thank you for responding.

I would say that as far as the value of BA's go it is really up to you and the research that you are doing as to their real value, not other donors. Though the BA donors are not a large segment of F@H would you really want to alienate them more by cutting BA points. They have spent allot of money in hardware and electrical cost to help in your research.

And I would ask the question of the people that have made the point that BA's are overvalued; do they actually run BA capable hardware? Remember that we have also run non-BA hardware in the past and the present.

[Viper97]

CRASH...

[Adak]

Closing the disparity between folding SMP and BA on the same rig, should be done. BA wu's are slightly overvalued. A 5% trim off BA points would help, imo.
The majority of the disparity - which goes as high as 67%, should be done by increasing SMP points by approximately 30% . That will leave us with an approximate maximum 32% drop off between a rig folding BA, and the same rig folding SMP projects, and less than that on several SMP projects. I believe that is where the points should be.

@troy8d, I noticed the 17% drop off between SMP and BA points. What system produced those? Bill1024 had far different data:

(This data was previously posted by Bill1024. It has not been independently verified)

Code: Select all

24 Core AMD 2.1GHz:
  8101 - 64% on 8558  36% less
  8103 - 43% on 8562  57% less
  
48 Core AMD 1.7GHz:
  8101 - 65% on 8560 35% less
  8103 - 37% on 8561 67% less
  8104 - 37% on 8802 67% less

SMP points and BA points should change in synch, so SMP points are always about 30% less than BA points, on the same BA capable benchmark PC.

@orion: Yes, I fold BA. Some posters in this thread are not BA folders, and some others are not currently folding at all.

[7im]

Closing the disparity between folding SMP and BA on the same rig, should be done. BA wu's are slightly overvalued. A 5% trim off BA points would help, imo.
The majority of the disparity - which goes as high as 67%, should be done by increasing SMP points by approximately 30% . That will leave us with an approximate maximum 32% drop off between a rig folding BA, and the same rig folding SMP projects, and less than that on several SMP projects. I believe that is where the points should be.

@troy8d, I noticed the 17% drop off between SMP and BA points. What system produced those? Bill1024 had far different data:

(This data was previously posted by Bill1024. It has not been independently verified)

Code: Select all

24 Core AMD 2.1GHz:
  8101 - 64% on 8558  36% less
  8103 - 43% on 8562  57% less
  
48 Core AMD 1.7GHz:
  8101 - 65% on 8560 35% less
  8103 - 37% on 8561 67% less
  8104 - 37% on 8802 67% less

SMP points and BA points should change in synch, so SMP points are always about 30% less than BA points, on the same BA capable benchmark PC.

@orion: Yes, I fold BA. Some posters in this thread are not BA folders, and some others are not currently folding at all.

Since BA points account for things other than a simple benchmark, like more memory, higher core counts, much larger work units, and so higher bandwidth usage, faster internet connections, etc., why should SMP ever line up with BA when SMP only needs a few cores and a GB of memory to run?

[7im]

And I would ask the question of the people that have made the point that BA's are overvalued; do they actually run BA capable hardware? Remember that we have also run non-BA hardware in the past and the present.

I don't have any female hardware either, but I can still make value judgments about women getting equal pay for equal work.

What does it matter if you run BA hardware when the more commonly suggested solution is to artificially raise the SMP points to match the already inflated BA points?

Seems like the 99% of donors running SMP work units should have more say about the SMP points than the 1% of BA folders not running SMP work units.

I'd also say PG has more to say about it than any one, based on their needs. If PG needs more SMP WUs done, and not so much BA work done, raising the core count does accomplish that goal to some degree.

[Bill1024]

Closing the disparity between folding SMP and BA on the same rig, should be done. BA wu's are slightly overvalued. A 5% trim off BA points would help, imo.
The majority of the disparity - which goes as high as 67%, should be done by increasing SMP points by approximately 30% . That will leave us with an approximate maximum 32% drop off between a rig folding BA, and the same rig folding SMP projects, and less than that on several SMP projects. I believe that is where the points should be.

@troy8d, I noticed the 17% drop off between SMP and BA points. What system produced those? Bill1024 had far different data:

(This data was previously posted by Bill1024. It has not been independently verified)

Code: Select all

24 Core AMD 2.1GHz:
  8101 - 64% on 8558  36% less
  8103 - 43% on 8562  57% less
  
48 Core AMD 1.7GHz:
  8101 - 65% on 8560 35% less
  8103 - 37% on 8561 67% less
  8104 - 37% on 8802 67% less

SMP points and BA points should change in synch, so SMP points are always about 30% less than BA points, on the same BA capable benchmark PC.

@orion: Yes, I fold BA. Some posters in this thread are not BA folders, and some others are not currently folding at all.

Since BA points account for things other than a simple benchmark, like more memory, higher core counts, much larger work units, and so higher bandwidth usage, faster internet connections, etc., why should SMP ever line up with BA when SMP only needs a few cores and a GB of memory to run?

Because a few cores will net you 7,000 PPD and no one want to run them, plus there is a backlog of 180,000 wus that need to be folded.
And server class hardware is not cheap nor is the electric to run them. Just a guess.

BA work units use less than 4 gb of memory and most computers today have at least 4gb. I would also say a majority of people have cable, DSl or Fios or other broadband connection.
This tread is about moving server class hardware from BA to SMP and right or wrong, the donors do not look happy about it.

[Nathan_P]

I'd also say PG has more to say about it than any one, based on their needs. If PG needs more SMP WUs done, and not so much BA work done, raising the core count does accomplish that goal to some degree.

How? When most of the affected donors are going to shut down their rigs? If PG need SMP folding to clear the backlog they only have to ASK. PG need to make a decision with BA, Either they allow 2p machines to contribute or they don't. The current plans would indicate that they don't want them. Oh and none of this "reasonable time" nonsense, my slowest rig can fold & send back a BA WU in less time than most SMP WU get finished.

Lots of good suggestions have been made in this thread - now its time for PG to show that they do actually listen to donors otherwise Mr/Mrs/Ms Donor relations guy is going to spend the 1st 6months of their tenure trying to mend a lot of broken fences.

[orion]

Lots of good suggestions have been made in this thread - now its time for PG to show that they do actually listen to donors otherwise Mr Donor relations guy is going to spend the 1st 6months of his tenure trying to mend a lot of broken fences

I do hope it's a gal because she'll have more sway over a bunch of geeky guys then a guy does.

Besides, PG already has a bunch of guys doing it for them now and look where it’s gone.

[Nathan_P]

Lots of good suggestions have been made in this thread - now its time for PG to show that they do actually listen to donors otherwise Mr Donor relations guy is going to spend the 1st 6months of his tenure trying to mend a lot of broken fences

I do hope it's a gal because she'll have more sway over a bunch of geeky guys then a guy does.

Besides, PG already has a bunch of guys doing it for them now and look where it’s gone.

Ha Ha, I did think about being gender neutral but the sentence just didn't seem to feel right.

All edited now

[Viper97]

The last post I made regarding the number of active machines currently working FAH was 226,224.

Today the number is 224,908. That's another 1316 machines offline.

The attrition rate is something to be worried about.

The crash continues.