STATS system down?

[Synate.Deszeld]

From an earlyer post stats was down not sure if they still aare maybe press or mod will post

[noorman]

.
At EOC, still no Stats coming trough; they are synchronizing their Stats now and the Result is still ZERO !

That 's the 6th three hour data block with no new figures

.

[shdbcamping]

Not asking for a miracle here . Just an update would be nice. Some of us have to judge their HW Science thing through these updates.

How can we know the HW is still running PG's WU's without them?

[VijayPande]

I kind of find it amazing that a group that has millions of dollars of donated computer resources to it can not keep the reward system up and going.

My team has been doing what we can without this support. I expect we will have this up shortlly. Sorry for the delay -- I more than anyone else wish we had the staff to do this more quickly.

Please understand that we don't have 24x7 staff to handle all issues and this happened on Saturday morning. If you would like to help fix this, please consider donating funds:
http://folding.stanford.edu/English/Donate
right now, we don't have enough funds to support 24x7 support, but once we do, I would be more than happy to support it. Note that 24x7 support requires about $1.5M or so every year, since 24x7 support means 3 people for the job normally done by one, the late shifts costs more money, and we have at least 2 FTE spots to fill (one for sysadmin and one for science/FAH infrastructure).

[Synate.Deszeld]

For 3 people 1.5 million? Gees thats a lot of money. I always estamated round the 100's of thousands range. Maaaan i was waaay off

[RAH]

Specially when you consider the work they do.

Well it to save money, what if we all shut down our computers at 5 pm PMT and turn them back on
Monday 8 am PMT. Wonder how much money and electricity would be saved on a global scale.

[VijayPande]

For 3 people 1.5 million? Gees thats a lot of money. I always estamated round the 100's of thousands range. Maaaan i was waaay off

it's 2 FTE's x 3 people = 6 people (for 24x7 support). Good sysadmins in the valley charge $100K to $150K in salary. Add overhead, benefits, etc almost doubles the amount. There are a lot of details like this you have to consider when running a project like this.

Please keep in mind that we've been running FAH for almost 10 years, winning award and award with ground breaking scientific results. If there were any simple, easy fixes, we would have done them long ago. I'm always to hear about creative solutions to fix problems we haven't solved (I think we've gotten the easy fixes, but I could be wrong or someone could have great idea), but this suggestion isn't one of them unfortunately.

I'm sorry for the delay on this one. You may have not have noticed my post above -- this looks now to be fixed.

[Synate.Deszeld]

OOOH Kool I did not know that. Live a lil learn alot. & in response to shutting our computers down i respect that BUUUUT they are submiting data & the ps3 is not. if people around the globe shut down there F@H computers at there home would that not harm F@H even more?

[kiore]

Thanks Prof Pande for the update.
So what we are missing are just the points scores, work is still being issued, completed and will be receiving points, we just can't see them yet.
The work servers are more important, the points will catch up, everyone is suffering from this equally at present.
Addit, although I'm a stats junky I'd rather see resources going to get the PS3 servers up first.

[kromberg]

Note that 24x7 support requires about $1.5M or so every year, since 24x7 support means 3 people for the job normally done by one, the late shifts costs more money, and we have at least 2 FTE spots to fill (one for sysadmin and one for science/FAH infrastructure).

A redundant failure over system would cost significantly less than that.

[VijayPande]

Note that 24x7 support requires about $1.5M or so every year, since 24x7 support means 3 people for the job normally done by one, the late shifts costs more money, and we have at least 2 FTE spots to fill (one for sysadmin and one for science/FAH infrastructure).

A redundant failure over system would cost significantly less than that.

Unfortunately, it's not that simple. This issue would not have been fixed by a fail over system (we have one). Please see my post above about "easy fixes" and consider that we have considered many such solutions, implemented the ones that make sense.

Finally, please also consider this is *just* for the stats system. The science/WS have been chugging along just fine.

Sorry for the delay on this one. But how about we move on -- it's working again and the 3rd party stats should be updating soon! (and the PS3 servers do seem to be up as well)

[shdbcamping]

I kind of find it amazing that a group that has millions of dollars of donated computer resources to it can not keep the reward system up and going.

My team has been doing what we can without this support. I expect we will have this up shortlly. Sorry for the delay -- I more than anyone else wish we had the staff to do this more quickly.

Please understand that we don't have 24x7 staff to handle all issues and this happened on Saturday morning. If you would like to help fix this, please consider donating funds:
http://folding.stanford.edu/English/Donate
right now, we don't have enough funds to support 24x7 support, but once we do, I would be more than happy to support it. Note that 24x7 support requires about $1.5M or so every year, since 24x7 support means 3 people for the job normally done by one, the late shifts costs more money, and we have at least 2 FTE spots to fill (one for sysadmin and one for science/FAH infrastructure).

Vijay,
is it a HW issue that is at the "ROOT Cause"???? I have on several occassions offerred support to get this rectified with absolutely no response. So fold I do... Donate or raise money for donations I will not. I will certainly be MUCH more conservative on WHERE some 'much more considerable' funds in donations would be concerned.....

Quite personnally, Why would one that is an outcast 'consider' throwing money at the overhead of this project when the attitude is " you are lucky we let you contribute"

I think a discussion of monetary need for the project would be a good "separate thread". I'm talking NOT the Mods/ admin from the forum as they tend to be way too petty and uninformed as to the project needs. I'm sure there are Donors that could do a Buck or two if they knew what/where it was going. Get a PayPal account for acceptance of donations.

As for myself, I'm gunshy about donating to a "donor unfriendly" project... only by my experience with having ALL my posts being censored/deleted by Mods... Bruce in particular.

Let's get with the Capitalist system here. Give a product that can be embraced after the (mod/admin) customer support experience, and you've got a small but steady input for the HW end of the BUISINESS of SCIENCE.

Again JIMO...YMMV.

[PantherX]

Thanks a lot Dr. Vijay Pande for updating us.
Personally, I believe that even when the stats are down for the third party, everybody is loosing the same amount (remember that you are getting 0 credits) hence your position in the 24 Hour Average would be same. However when the update happens, thinks will change as there will be a huge amount of points to add. (I am guessing but if anyone good in maths wants to verify it, please do it:).

Right now, the Official Stats for me has updated with the correct amount of WUs!

[noorman]

A redundant failure over system would cost significantly less than that.

Unfortunately, it's not that simple. This issue would not have been fixed by a fail over system (we have one). Please see my post above about "easy fixes" and consider that we have considered many such solutions, implemented the ones that make sense.

Finally, please also consider this is *just* for the stats system. The science/WS have been chugging along just fine.

Sorry for the delay on this one. But how about we move on -- it's working again and the 3rd party stats should be updating soon! (and the PS3 servers do seem to be up as well)

.


Thanks for the fix and the update, Prof. Pande

.

[shdbcamping]

1st, Vijay, don't get to this level of arguement as it degrades yourself to others . Your project is doing very great stuff for science. The fact that you are apparrently lashing out at a "DONOR" with a byach to pitch ATM is not doing the "Project" any good.. Leave that to bruce and the 'minions' as they are more than capable .

2nd: I believe the issues are much more driven by HW issues than IT. OR does not Stanford have an IT program?????????? If so I'm compressed. Have we not considerred the same 1st line of Students that this project is using for the writing of the WU's? Most if not all of these overwhelming problems could easilly be resolved by enlisting some interns from Stanford in a 'directed' project, would it not?

3rd: Engage a Donor opportunity... with specifics on where the money goes and get a separate and better recieved way of keeping the $ vs donor thing going. This 'help forum' chases away more 'Donors' than it contributes to.... JIMO and as always YMMV

Sean