Stats Server Down

[snapshot]

Thanks. Will do.

[rafwiewiora]

Re: spikes in points - choderalab is all good, I had 150 GPUs on 11730 in internal testing.

[EOC_Jason]

Okay, I managed to finish (what I thought) was everything before the 3PM update so I let things run as scheduled. There looks to be a few names that are still duplicates due to some special character issues, but after the 6pm update I'm going to run a script that will find any users in the DB that aren't in the flat-file and remove them. It's probably going to be just a small handful of people (double digits at most), most aren't even active. There was one that had a lot of points and I fixed his account manually so he wouldn't lose all the historical data. Also I had a little oops in the Team 0 processing that wiped out & re-created some of those users, but I've fixed that bug. Can't do anything about that lost data...

I'm going to see what happens with the next update (in about 30 minutes) and tweak as necessary. But for the most part things should be back to normal.

I've got some other little code tweaks related to processing to do this weekend, but it shouldn't affect anything.

[snapshot]

Re: spikes in points - choderalab is all good, I had 150 GPUs on 11730 in internal testing.

Thanks for the info. That must have been fun.
I can easily accept one team doing this but another one to the same extent plus the five lesser ones I've found in the same narrow window seems unlikely. It may just be that I'm paranoid but it still looks odd especially as Jason has finished his updates and nothing has changed.

[DutchForce]

It looks like the Stats server is not updating.

I'm also getting an "Error failed to get project data" when I go to the Project Summary and no servers are showing up on the Server Status page.

[gordonbb]

Ditto,

@bruce @sortofageek @Macaholic @susato @Ivoshiee @toTOW @Joe_H @ PantherX
The stats servers reported about 1/2 the expected points for both myself and my team at the 15:00PST Update and now at the 18:00PST update is reporting 0 points for both.

No servers are showing in in the Server Stats but the individual stats are still showing values they are just not current.

[JimboPalmer]

Just before a long holiday weekend in the US, that server has it down pat.

[gordonbb]

Just before a long holiday weekend in the US, that server has it down pat.

its failure is exhibiting different behaviour though. It is still producing the flat files but those flat files are not being updates from the collection servers.

I’m still being assigned work units and the completd jobs are being accepted so it’s all good.

I’ve worked in academic environments for over twenty years and understand that support is, of necessity, “best effort and running code”

Perhaps the stats server just needs and extended long weekend before mid terms to spend some time chillin’ with family

[drougnor]

It looks like the stats 'froze' starting at the 1PM EST update for Users flat file and 2PM EST for the Teams flat file. The time/date stamp is updating, but the numbers for both the individual users and teams have both frozen.

d

[bruce]

I'm also getting an "Error failed to get project data" when I go to the Project Summary and no servers are showing up on the Server Status page.

Fixed.

[goodyca]

The stats have stopped being updated. The last update occurred at 5:45 GMT today.

[fluke76]

The stats have stopped being updated. The last update occurred at 5:45 GMT today.

Yep. I noticed that, too. I have at least 1 WU that was successfully uploaded, but not updated on Stat server. It is a US holiday so that might have something to do with it. If you have been folding for a while, you probably know that the Stat server thinks it gets a day off for a holiday.

[gordonbb]

The stats have stopped being updated. The last update occurred at 5:45 GMT today.

Yep. I noticed that, too. I have at least 1 WU that was successfully uploaded, but not updated on Stat server. It is a US holiday so that might have something to do with it. If you have been folding for a while, you probably know that the Stat server thinks it gets a day off for a holiday.

LOL or the support folks (likely grad students) do

[Murford]

Mod note: Moved to existing topic on stats being down, added Code tags to your log - j

I have been running F@H for about a week without issue. I noticed yesterday that I received no credit, but when I checked the log there were completed WUs with estimated credit listed. I restarted my computer to see if it was an issue on my end, but it seems to still be happening. Restarting reset the log so I don't have the logs from yesterday, but I do have a log of a completed WU from today that it looks like I haven't been credited with.

[Joe_H]

As I posted here - viewtopic.php?f=16&t=31338&p=304877#p304877 - I have notified the stats site maintainer.