Is it just me or are the Stats not updating?[Nov 12 problem]

[TheWolf]

I was hoping to see a few more of my lost work units show up today, like yesterday.
Didn't happen.... looks like I got about 15% of what's missing from the down time.
Maybe tomorrow will bring a few more xx%.
Actually I was shown a little short today, norm is 50k to 53K, only 47k the past 24hrs.

[Mactin]

It would be nice if PG kept the community better informed.

[7im]

What parts of their statements in the News was not clear?

Nov 12 : http://folding.typepad.com/news/2009/11 ... on-it.html

Nov 23: http://folding.typepad.com/news/2009/11 ... gress.html

[noorman]

.

It 's about the fact that the update of 23 Nov. doesn't seem to be the 'whole lot'; many have seen a partial update (as did I).

We 're just wondering when the 'rest' is coming. I saw missing data on 2 following days; I guess that they did a correction for the one day and are working on the second day's data now ...


.

[Grandpa_01]

I dont think it was not being clear part at all. The statement was a bit vague would it have been that hard to say we have a certain % done and you can expect to see more by ???. PG has a bit of a problem about being vague with ther statements. I have yet to even seen it mentioned where there still may be a problem with the stats server. Yet several people have mentioned including myself that their daily production WU / Point count seems to be coming up short. So is PG getting all of the WU's that I am completeing (I don't know ) but it does't appear to be.

[7im]

The data wasn't lost, it's just a matter of time to run through all the data and recredit the points. Remember, this is a manual process, not a computer process. That means a human sits down at a terminal and does the work. And the bigger the project gets, the longer these things take.

And sometimes it's easier to be vague than to have to craft a politically pleasing statement to the affect that we should be more patient, and that they will work on the problem as time permits. I'd have to guess that a donor's priority level for recrediting is much higher than it for Stanford, no offense, but they have families and holiday plans as well.

[noorman]

.

For hours now, I haven't been able to gain access to my personal stats on the Stanford website either (nor those of fellow Folders) ...

So, there seems to still be something wrong with the Stats, unless they are now running another update cycle which entails a blocking of public access to the official Stats. I always get this known message "Stats update in progress" !

EDIT: I 've, just now, gotten access to my stats


.

[noorman]

The data wasn't lost, it's just a matter of time to run through all the data and recredit the points. Remember, this is a manual process, not a computer process. That means a human sits down at a terminal and does the work. And the bigger the project gets, the longer these things take.

And sometimes it's easier to be vague than to have to craft a politically pleasing statement to the affect that we should be more patient, and that they will work on the problem as time permits. I'd have to guess that a donor's priority level for recrediting is much higher than it for Stanford, no offense, but they have families and holiday plans as well.

.

I can't imagine though, someone doing the stats of every uploaded Result over at least a 24 hr period ...
I guess there may be manual labour to it, but not as tiresome as that, surely ?


.

[bruce]

Well, I think it probably goes something like this.

Consider how many WUs were submitted during the time that the stats were down. (Be sure to include some time before and some time after, so that you're sure to get all of them.) Now consider how long it takes to collect that information from whatever backup sources are available for each work server. Now take that massive list and compare it against the list of WUs that you've already gotten credit for to eliminate duplicate credits. (Be sure to consider the size of the list of WUs that everybody has gotten credit for, too.)

This is a manual process, so your primary tools are "grep" and SQL queries -- oh, and you have to be VERY VERY careful, because every donor will be complaining if there's a mistake - - oh, and you have to be VERY VERY fast, too, because every donor will be complaining that you're not fast enough.

Yes, it's every bit as tiresome.

[noorman]

.

OK, I got it; they could have told us that this wouldn't be a one-go update (even immediately after the fact)

That would have negated the need for all these questions and guesses ...


.

[7im]

Haven't you been folding long enough to know that ALL of these manual updates areN'T a one-go? Nothing has changed since the list time...

[TheWolf]

Haven't you been folding long enough to know that ALL of these manual updates are a one-go? Nothing has changed since the list time...

Don't you mean "aren't" a one-go thing?

But yea I've never seen all my credit show up all at once.
Normally around a week to see them all.
Since the folding community has grown "probably doubled" since the last problem,
I'm sure its going to take more time now than ever before.
Looks like up to 3 weeks, since 2 weeks has almost pasted.

[noorman]

Haven't you been folding long enough to know that ALL of these manual updates are a one-go? Nothing has changed since the list time...

.

I 'm not with you; many thought that the update would be all the backlog restored in one go, but it doesn't seem to be the case.
In the past, I can't remember having seen updates that were done in several slices



.

[road-runner]

I dont know if I got all mine or not and really dont care as long as they got they work, life goes on...

[Gary480six]

Sill not seeing any missing points on my end (only missing about 21k). I understand that it must be an arduous task and our individual/team points are hardly the focus of the project so please don't take this as a complaint. I'm just wondering if there is any new info.

Edit: I'm also wondering if there are any remaining issues with the stats/credit. I switched a couple of multiple standard CPU client setups over to single SMP clients with an increase of couple thousand PPD (according to FaHmon) and yet my daily credits have been slightly lower since making the changes. I have confirmed no EUE's and all log files are reflecting successful uploads on all clients.

I have to agree with J_M. I also feel that since the crash, the stats/credit server has been missing some current credits. I'm guessing that I've lost about 5000PPD. But since I complete about 80 work units a day, I cannot figure out if I'm being shorted with the SMP or GPU clients - Or if the Passkey (or not) has something to do with it.

Anybody else see an obvious drop in their PPD since the crash was "fixed"?