How long is the data about old WU's retained?

Moderators: Site Moderators, FAHC Science Team

Post Reply
ETA_2025
Posts: 105
Joined: Mon Jan 30, 2023 10:43 am
Hardware configuration: NVIDIA RTX 4070
20 x Raspberry Pi 5 Model B 2GB RAM
Location: VIC, Australia

How long is the data about old WU's retained?

Post by ETA_2025 »

How long is the data about old WU's, including those credited, dumped, failed, pause(d) or rejected retained?

I've got 26 days worth of data, which is how long 8.4.9 has been installed on my various machines.

Will it be purged at some stage, or will I need to do that manually if so desired?
Image
daiko
Posts: 24
Joined: Tue Jun 08, 2021 1:14 pm
Hardware configuration: Mac Studio M1 Max
Mac Mini M4 Pro
Folding since 2005
Location: Atlantic County, NJ

Re: How long is the data about old WU's retained?

Post by daiko »

I'm seeing 167 days worth of work under my Units tab, less than 6MB of log files.
So that would be about 12MB/year for this CPU WUs-only machine. So I’m not too concerned personally, but if you’re doing GPU work, I could see this eventually becoming a problem?
Image
muziqaz
Posts: 1426
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:

Re: How long is the data about old WU's retained?

Post by muziqaz »

The limit is hard set to 500 entries per machine. I tried negotiating for more, no luck, since after 500 entries in the db file performance degrades considerably, I am told.
This is about the v8 Work Units section
FAH Omega tester
Image
ETA_2025
Posts: 105
Joined: Mon Jan 30, 2023 10:43 am
Hardware configuration: NVIDIA RTX 4070
20 x Raspberry Pi 5 Model B 2GB RAM
Location: VIC, Australia

Re: How long is the data about old WU's retained?

Post by ETA_2025 »

muziqaz wrote: Sun Feb 16, 2025 12:24 pm The limit is hard set to 500 entries per machine.
Thanks muziqaz
Image
rbpeake
Posts: 142
Joined: Sun Jun 15, 2008 4:39 pm
Hardware configuration: Intel® Core™ 2 Duo processor E8500, dual 3.16GHz cores, 6MB L2 Cache, 1333MHz FSB (45nm); 4096MB Corsair™ XMS2 DDR2-800 RAM; 256MB eVGA™ NVIDIA® GeForce™ 8600 GT Video Card
Location: NYC Metro Area

Re: How long is the data about old WU's retained?

Post by rbpeake »

Is there a way to delete the data on old work units before 500? Does it affect processing time?

Thanks.
muziqaz
Posts: 1426
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:

Re: How long is the data about old WU's retained?

Post by muziqaz »

rbpeake wrote: Thu Feb 27, 2025 9:05 pm Is there a way to delete the data on old work units before 500? Does it affect processing time?

Thanks.
No, and no :)
FAH Omega tester
Image
whocrazy
Posts: 97
Joined: Thu Mar 27, 2008 9:09 pm

Re: How long is the data about old WU's retained?

Post by whocrazy »

When FAH isn't running, you can try running a vacuum operation on the client.db file. sqlite3 client.db vacuum.
muziqaz
Posts: 1426
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:

Re: How long is the data about old WU's retained?

Post by muziqaz »

whocrazy wrote: Sun Mar 02, 2025 4:28 am When FAH isn't running, you can try running a vacuum operation on the client.db file. sqlite3 client.db vacuum.
Let's not suggest these type of operations, please
FAH Omega tester
Image
arisu
Posts: 176
Joined: Mon Feb 24, 2025 11:11 pm

Re: How long is the data about old WU's retained?

Post by arisu »

muziqaz wrote: Sun Mar 02, 2025 8:39 am
whocrazy wrote: Sun Mar 02, 2025 4:28 am When FAH isn't running, you can try running a vacuum operation on the client.db file. sqlite3 client.db vacuum.
Let's not suggest these type of operations, please
What would go wrong with a vacuum operation? Unless I am mistaken about sqlite3, the only time a vacuum operation can go wrong is if the database is already seriously corrupted (it would just make it worse although that could be said for any write operation).
muziqaz
Posts: 1426
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:

Re: How long is the data about old WU's retained?

Post by muziqaz »

Let's not suggest outside actions to the files run by fahclient. Period.
FAH Omega tester
Image
arisu
Posts: 176
Joined: Mon Feb 24, 2025 11:11 pm

Re: How long is the data about old WU's retained?

Post by arisu »

It's a regular sqlite3 database, whether or not recommending doing things is a good idea in general, vacuum should be a safe operation.

Is your recommendation just a matter of leaving things alone because some people may not understand which operations are safe and which are not? Since it can grow pretty large, I might add a ticket to the tracker to add the auto_vacuum pragma to the database.
calxalot
Site Moderator
Posts: 1406
Joined: Sat Dec 08, 2007 1:33 am
Location: San Francisco, CA
Contact:

Re: How long is the data about old WU's retained?

Post by calxalot »

There is already a ticket suggesting vacuum
It is old. Someone should probably offer a PR.
I think it would be a one line fix to add the pragma auto.

https://github.com/FoldingAtHome/fah-cl ... issues/123
calxalot
Site Moderator
Posts: 1406
Joined: Sat Dec 08, 2007 1:33 am
Location: San Francisco, CA
Contact:

Re: How long is the data about old WU's retained?

Post by calxalot »

muziqaz
Posts: 1426
Joined: Sun Dec 16, 2007 6:22 pm
Hardware configuration: 9950x, 7950x3D, 5950x, 5800x3D
7900xtx, RX9070, Radeon 7, 5700xt, 6900xt, RX 550 640SP
Location: London
Contact:

Re: How long is the data about old WU's retained?

Post by muziqaz »

arisu wrote: Wed Mar 05, 2025 7:48 am It's a regular sqlite3 database, whether or not recommending doing things is a good idea in general, vacuum should be a safe operation.

Is your recommendation just a matter of leaving things alone because some people may not understand which operations are safe and which are not? Since it can grow pretty large, I might add a ticket to the tracker to add the auto_vacuum pragma to the database.
The reason I'm saying that is that regular people use this forum, not fah Devs, with some exceptions. So you might know what you are doing, but there might be some people who think that your suggestion is a solution, and use it, and break things, however unlikely the breakdown might be. And then we have to listen and troubleshoot those people's problems.
Any low level suggestions should be done in GitHub. Dev resides there and you will get most attention to your suggestions.
The deletion of dumped, crashed WUs have been already suggested in GitHub few times. Latest one might have even been mine.
FAH Omega tester
Image
Post Reply