Folding Forum

I figure this topic should be brought up, not for fighting, not pointing fingers, not causing an argument, but as a discussion to figure things out.

Back in October, I posted a thread about failing work units in large volumes. You can look it up and see the end result if you wish. When I posted, I did it out of genuine concern trying to help the community so that we could find a cure for the proteins that we fold to find a cure...

Immediately, fingers went pointing to bad hardware.. Trust me, the posts are still there. We showed our hardware was fine.. And fingers were pointed blaming us for not believing it was the hardware.

What ever happened in the end, while these forums pointing at bad hardware? Stanford found the bad code that caused the failures.. Did ANYONE from this hardware admit their fault? Did they admit it as NOT the hardware? Nope.. They vanished from the thread the moment that I posted the article from Stanford showing that the work units were the problem the ENTIRE time.


Does this forum TRULY want to lose this program? Does this forum really care about the folding program at all? Do you want the power house folders to walk away 90% of people avoid these forums because of the saddening finger pointing and extreme childish bans.

When we have PRODUCED results by complaining about failed work units, and we produced proof from Stanford that it wasn't our hardware, the forum went silent.. No apology, not single message saying that it was good Stanford found that They had an error in their coding.. Nothing. Silence.

We are abusing thousands of dollars worth of hardware per user for Stanford (billions of dollars world wide), and we would like to be allowed to ask questions and make statements when there are MAJOR issues at hand. If you feel you are being attacked, step back and look at the bigger picture.. We aren't risking our hardware, abusing our hardware, or folding for this forum. We are trying to reach out for help and find out why failures are happening and get better results so that we as a community can produce results.

I will say, when we as users start reporting a trend, it should be passed along without the judgemental attitude. When thousands suddenly experience an error, pass along the info. Reassure that it is either being looked into or solved. It's really simple. We don't have access to Stanford, because they don't care about their folders.. It is free hardware and research to them, or they would give us access to ask the questions. If everyone stopped folding, Stanford would have nothing.. Yet, they get funding for research off of our completed work units.

I want to thank the naysayers for making our entire team as unwelcome as humanly possible. I guess it pays to be part of the number 1 team by a ridiculous long shot.

In 1997, my mother was diagnosed with cancer of the brain.
The Doctors assumed it had undergone metastasis from some other part of the body. They spent a considerable amount of time looking for that other cancer, for a couple of reasons: You can biopsy other parts of the body easier than you can the brain, and under 10% of brain tumors start in the brain.
Sadly, my mother had that 10% case, and they were never able to do a biopsy without killing her, and so were never able to get a treatment plan. While unhappy she was terminal, at no point did the Doctors apologize for investigating the best case, most obvious scenarios first. The cases where it originated in the brain had extremely low patient outcome and required help from some other source.

This resembles your scenario, the folks here investigated and pursued overheating and driver incompatibilities because those are very common and can be solved with the resources here. (There is another site for programming bugs, errors, and features. https://www.reddit.com/r/foldingathome/ ) While that was not the case in your situation, it fit your symptoms. (at least initially)

Our symptoms were bad coding, and through "whining" we made it obvious that thousands of work units were failing because of it. While we "whined" this forum pointed at our hardware. Even Stanford provided clarification later that it was bad coding.

I could be misunderstanding this, but comparing bad moderation to terminal cancer is 100% accurate. It will kill the program and has already removed hardware from the field.

If Stanford would give me access to check on the statistics that are always shown as "well,someone else on your team completed it, so it's your hardware" then I would gladly devote time to helping the masses and reporting the issues.

Unfortunately, reddit is ignored. I refuse to use reddit as it is a troll fest over there.

Our concerns, that Stanford makes money off of, should be answered by Stanford. Stanford wants free support, they should show gratitude.

The problem with first released FAHCore_21 was that some work units worked fine and some work units failed with "bad state".
Later a software problem with PME was found and that was the reason and the next released FahCore_21 fixed it.

The users posting in this forum may be 14 years old playing with their GPU overclocking or 40 years old and manage a small server farm.
When they have problems we try to narrow down if FAHclient is setup fine and hardware is working stable or there may be a driver problem.
We cannot fix software errors this is done by Pande Group (10 guys at standford university trying their best).
And we didn't know if the users posting have a hardware problem or the software has a bug.

In your case maybe your writing style can seem a little rude to some readers.
So me e.g. did not have the maximum motivation to further discuss your demandings.
But maybe I do not feel right about your postings and so I apologize if I did a misunderstanding.
I hope you will stay a valuable folder for the future.

The initial responses to the posted problem put everyone's back up and it just got more heated from there, I think everyone who posted in that thread needs to take an objective look at the thread, especially their own posts, and think about how they came across and how their posts may be perceived by others and whether the post could have been worded differently. Very few of us actually know anyone else in person, making it very hard to put a context on what is being said and how a post reads.

I will say as well that donors on here especially and to the program in general are treated very poorly, their efforts and expense in donating are very undervalued.

foldy wrote: In your case maybe your writing style can seem a little rude to some readers.
So me e.g. did not have the maximum motivation to further discuss your demandings.
But maybe I do not feel right about your postings and so I apologize if I did a misunderstanding.
I hope you will stay a valuable folder for the future.

My writing style is indeed considered rude or straight to the point. That depends on who is reading it and how they want to take it.

I did not know there was only 10 people at pande group working on this project, and I appreciate their efforts and would love to see a short monthly update. It could be a single sentence saying this has been a great month, or a paragraph explaining how to diagnose an issue, or that they are looking into recent failures.

Since my initial posting, and the finding of the error in the coding, I have had one bad work unit that I have spotted. That doesn't mean there was only one, but it was the only error reported when I did look.


I personally won't stop folding because of a different forum not treating me the way I want. I have taken a break since I am out of state and can not monitor my hardware at the end of the day. I would just simply like to see a little better treatment for everyone.

No one knew this was bad coding initially. And since some GPUs could successfully complete the exact same work units that you reported as failed, the most common diagnosis of the problem was hardware or driver related (and has nothing to do with moderation, it just happened to be posted by mods). It was assumed that if the code was bad, that all GPUs would fail, much like the doctors above assumed the 90% of cancer starts in the body. Also true for bad code, it fails on most GPUs most of the time.

So yes, you misunderstood what JimboPalmer was relating. He was not comparing cancer to bad moderation. The process of diagnosing a WU failure is like the process of the doctors looking for the cancer. They both follow a standard well considered diagnostic process. Again, answers posted by Mods is not the same as moderation. And Non mods were posting similar commonly used troubleshooting suggestions. Suggestions that worked well in the past.

Reddit is the tool provided. Moderated here too much, not moderated there enough. Just can't win. But feedback to PG in this forum is unlikely to be seen in any timely manner. Use Reddit or don't. This 3rd party run forum can't change that. We're here to deal with technical support issues, not political issues.

Stanford makes no money from folding, so that should ease your concern. And Stanford often expresses thanks to the general donor. Here is a blog post from Dr. Pande recently.

Summary. I’d like to thank all of our donors for their contributions to Folding@home. It’s exciting to me to be discussing these amazing results and 15 years ago would not have imagined that we’ve been able to do what we do now and I’m excited in terms of how our vision and plans for the future will be able to push us even farther.

I think what you are asking for here is a personal thanks for being persistent in looking for a good diagnosis to the Bad State problem. I agree. Thank you for being persistent. Thank you for continuing to chase down a problem when a lot of the standard Bad Work Unit diagnostic processes didn't point to the correct answer, and truly did point at a hardware problem (originally). Later trouble shooting obviously found the coding issue and PG fixed it. Your tone may have been a little off-putting and misdirected, like suggesting the moderation was like cancer in the above post. Moderation is the act of enforcing forum policy, it is not the act of trying to help diagnosing problems. And without the mods taking these types of issues to the developers, fixing this would have take much longer.

And like JimboPalmer's mom, I am sure she was thankful the doctors were persistent in looking for ways to help fight her cancer. I am also sure that when the normal diagnostic procedures didn't work, she didn't ask for an apology from the doctors. I am sure she thanked them, even after the terminal diagnosis. Thank you Scarlet-Tech. Just go easy on the doctors here. They are not Pande Group, and they are not well paid oncologists, but simple volunteers.

I apologize for whatever inconvenience you experienced during the debugging of the problems you reported, but I do not apologize for trying to help.

As has already been said by others, eliminating hardware is a necessary step in isolating a problem, and it seems particularly significant since you were the only one reporting the problem. Eventually we learned that the problem did occur on the latest hardware (GM2xx) but not on GM1xx and earlier) so lots of folks didn[t experience the problem.

Nevertheless, once hardware was eliminated, the discussion moved on. I don't think my questions were a waste of time even though it later turned out that I was wrong.

If some part of the discussion caused you to feel attacked, it would have been appropriate to speak with the offender privately or report to a moderator privately at the time. Private conversations would have provided opportunities to correct the situation then rather than waiting for months.

In that topic, you thanked everyone for their help. I suppose someone from Stanford (or a volunteer, like myself) should have thanked you for your report, too, but once problems are fixed, everybody tends to forget about them except those harboring a grudge.

Development was looking into the problem in their lab but those activities are not reported here until the issue is resolved. At that point we only had one report (yours). Stanford's lab has a mixture of GPU hardware, unlike your own which apparently was all GM2xxx.

Per Development policy, projects which seem to be problem free are moved to beta and perhaps on to advanced, but that doesn't GUARANTEE that they're actually problem free on all possible systems. Based on your reports, those projects were suspended until it could be identified and fixed. The only projects you saw were problem free in the lab.

- - -

The remainder of your post will not receive a public response in this forum. If you feel you have been moderated unfairly, you should initiate a private discussion with that moderator per the forum policies

Regarding the forum policies:

Could you update the appeal process? Vijay Pande has left Stanford and is currently General Partner with Andreessen Horowitz venture capital.
RE:
" If you are not satisfied with the Admin's response, you can then further appeal this to the Pande Lab directly, with Prof. Pande being the final arbiter of issues related to Folding@home. "

Who is in charge and the final arbiter of the project?

Dr. Pande is still the final arbiter of all issues related to Folding@home. He is no longer at Stanford, but he's still actively running the project and checks in on a regular basis and is responsible for future development plans as well as coordinating with the distributed labs which have been started by PG graduates.

In accordance with the policy, you need to start with the forum Mods.

I can understand that some failures can be attributed to failed hardware, but the incident in October was not limited to any one team, and it was certainly more wide spread than anything previously.

It has been months, as I stated earlier, since I received a failed unit that I have noticed. At that time, there was a 30% decrease in successful work units across many platforms, and entire team numbers greatly decreased, which should have pointed to more than just a hardware issue, at least in my eyes. Now, behind the scenes, there was research going to get the issue resolved.

I do appreciate them looking into that. I get home tonight and will be turning the machines back on for a few days before I go out of town again.