Logs and databases

[FoldingFodder]

Hi,

Are the assigned, timeout, and expiration epoch timestamps stored long term?

I see they're temporarily stored in the Units table of the client database whilst the core/client is working on the WU.

I'm surprised not seeing them in the Projects table since it has some history. Also surprised to see there is nothing stored in the description column, not even a generated link to a URI.

I've looked in databases, logs, xmls, jsons but can't find anything long term. So i'm wondering if i've missed the information or if it's else where.

[Neil-B]

Assigned, returned and credited are all stored server side for every WU result from every folder ... the timeout and expiration periods are recorded server side for every project as is the server information ... not sure why this would be specifically stored elsewhere - though certain elements of this are stored in logs and for people using either their own logging scripts (I scrape my logs for every WU PRCG and pull the info from WU stats to give me precise points counts and timings) or tools such as HFM this data may be historically cached client side by these.

Would some of the info in the projects table be there simply to help the client publish ETA figures? ... as such it wouldn't need to store links to projects descriptions just some minimal completion type info?

[FoldingFodder]

Assigned, returned and credited are all stored server side for every WU result from every folder ... the timeout and expiration periods are recorded server side for every project as is the server information ... not sure why this would be specifically stored elsewhere - though certain elements of this are stored in logs and for people using either their own logging scripts (I scrape my logs for every WU PRCG and pull the info from WU stats to give me precise points counts and timings) or tools such as HFM this data may be historically cached client side by these.

The client seems to store everything else other than these long term - some information is seen in multiple logs and the databases. Which is why it made me wonder if it was stored else where or whether i had not spotted it.

Would some of the info in the projects table be there simply to help the client publish ETA figures? ... as such it wouldn't need to store links to projects descriptions just some minimal completion type info?

There is a column specifically called description, which every entry is empty. The projects table is fairly bare with only a few columns.


My plan is similar to what you do. Collect what i want locally and then the remaining information from the servers.

[Neil-B]

I made sure I changed the logs to keep more than the last 16 - I set log-rotate-max to 99 (wasn't sure anything over this works) and lift them off the box.

Some fairly simple parsing of a log grep allowed me to create the URLs for the WUs I have folded and grab the information from the WU stats ... There are bits I could do better - I use my version of "estimated PPD" by using the days figure from the WU stats (which includes server side and upload delays - luckily I tend to have few of those) whereas I could parse out the true "folding time" for every WU and match that with the actual points awarded to get a much truer figure.

Most of the time I am really doing the stats just o keep my mind working a bit whilst in lockdown - they are reasonably useful for knowing if beta projects I am running are slower/quicker than norm and awarding a kind of expected amount of points ... As data goes I am happy to grab what is easy and find a way of creating the rest I feel I need