Am I getting too old/slow for FAH ?

[jives11]

Hi, I've been running FAH for a few years, mostly on 5 Macs , all pre 2014 (2 Pros, 3 minis). I sleep the machines during the day and they wake between 00:00-05:00 as my electricity cost is 1/5th the rate during those hours, designed for EV charging.

This was working well until I'd guess a few weeks ago when I noticed my active clients falling off

Active clients
5 within 50 days
2 within 7 days

In the night I monitored the machines via FAH control,and while all of them were happily crunching, I noticed none would meet their deadlines. I also noticed the size of the WU's were all much larger than previous ( 5 figure credit rather than 4) and the time slot seemed to still be the same as before i.e circa 4 days.

So is this a temporary workload blip that will sort itself out and I'll get back to nice achievable workunits, or is it time to stop. I don't want to waste people's time nor burn electricity for no good end. I could redeploy them to run boinc as many of the workloads are not as time critical ?

Is it time to retire ?

[bikeaddict]

It looks like three of your clients missed the deadline on recent WUs, according to https://apps.foldingathome.org/cpu

The work was then sent out to be processed by another client.

The two that returned WUs on time were extremely slow, taking 3-4 days each and did not earn the bonus.

Returning expired WUs is worse than doing nothing. Someone else ends up having to do the same work over again instead of processing the next available new WU. It further delays the return of the work to the researchers who have to wait days for results before generating new tasks based on the results. And you are wasting money on electricity and not doing useful work.

Hardware this old is too slow and inefficient to do work quickly and efficiently and probably should have been sent to the e-waste recycler a few years ago. There are probably computers being sent for scrap every day that are years newer and far faster and more efficient that could maybe still be used for distributed computing.

[jives11]

Thanks for the frank assessment. I wasn't aware of the web interface you linked to , I simply use the stats page. To find overdue WU's I have to look in the log files ia the client and scroll back, and this makes me wonder how many other people are unaware that they are returning units late ?

I have similar spec machines working on various boinc projects, for healthcare and astronomy , so I'll divert these machines to those. I have an RPi Arm32 Model 1 doing work earning credits on Astronomy projects. So I'll not be recycling these machines any time soon.
Not every problem needs to be solved by a Hare, sometimes a Tortoise is good enough

[bikeaddict]

It's more than just speed that's a drawback. Old hardware wastes power and costs money for the utility bill while doing very little work in the process.

[jives11]

That depends. Computers are about as efficient as any other space heater. Old hardware may be slow but it can do more research than a heating element. If you live somewhere on the cold side the waste heat is welcome. Our central heating only went off two weeks ago, though it has been a poor summer in the UK

[bikeaddict]

Well I care about doing the most scientific research possible for the money. That requires at least semi-modern, used hardware like Ryzen or Xeon CPUs with a lot of cores. Quad or dual core is just a waste of power for distributed computing. I've slowly gone from 4 to 6, 8, 14 and finally 16-core CPUs over the last four years.

[jives11]

There is no cheaper unit of cpu as a free one. As I said, the power issue is only an issue when waste heat is inconvenient or unwelcome .

Anyway I’ll deinstall FAH and not sully the points table with my aged hardware

[appepi]

I was inspired by this debate to document my own reasons for shutting down CPU processing on my elderly HP Z-series workstations, even though they have 1 or 2 6-core CPUs and thus 12 or 24 logical processors available.

I will start by documenting the Geekbench 6 (GB6) performance and Folding PPDs of those Macs that seem to fit the description by Jives11 (I have not had anything to do with apple devices since an Apple ][+ in 1983 so I am guessing to some extent. I will illustrate the data with the first group.

Mac mini (Late 2014)
Intel Core i7-4578U @ 3.0 GHz (2 cores) 1C=970 MC=1913 cf 884/1677
Intel Core i5-4308U @ 2.8 GHz (2 cores) 1C=947 MC=1862 cf 1001/1999
Intel Core i5-4278U @ 2.6 GHz (2 cores) 1C=879 MC=1750 cf 912/1810 cf 6275/25100 (4LP)
Intel Core i5-4260U @ 1.4 GHz (2 cores) 1C=739 MC=1387 cf 729/1454

There are various CPU's for those above, and GB6 has a separate series of benchmarks when they are in MACs versus other devices and OS's. (See https://browser.geekbench.com/mac-benchmarks versus https://browser.geekbench.com/processor-benchmarks.
Thus we can see that a Mac Mini from the (Late 2014) series with an i5-4278U CPU has a single-core (1C) GB6 benchmark of 879 in a Mac versus 912 over all devices and OS's, and a Multicore benchmarks of 1750 versus 1810.

If we now go to https://folding.lar.systems/cpu_ppd/overall_ranks , we find that this translates into 4 Logical processors (4LP) for max folding performance (with no concurrent GPU) 0f 6275 PPD per LP, and thus 4x6275=25100 PPD at most (this is how LAR Systems deals with the fact that the data that comes to them will vary in the number of LPs allowed to fold. They convert the PPD data to "per LP" and multiply by the max LP that MIGHT be used. [They also give data disaggregated by number of LPs, OS's etc, but that is not needed here except to note that output is higher with Linux than Windows, much to no-one's surprise.] Note that I could only find PPD data for 1 of the processors.

With that out of the way let me run through the other Macs and their sub-variants.

Mac Pro 6,1 (Late 2013)
Intel Xeon E5-1620 v2 @ 2.7GHz (4 cores) 1C=749 MC=2622 cf 643/2362 cf 3290/26320(8LP)
Intel Xeon E5-1650 v2 @ 2.7GHz (6 cores) 1C=761 MC=3565 cf 673/3334
Intel Xeon E5-1680 v2 @ 2.7GHz (8 cores) 1C=760 MC=4329 cf 658/4053
Intel Xeon E5-2697 v2 @ 2.7GHz (12 cores) 1C=679 MC=4896 cf 555/4301 cf 3829/91896(8LP)

Mac mini (Late 2012)
Intel Core i7-3720QM @ 2.6 GHz (4 cores) 1C=680 MC=2333 cf 599/2030 cf 6574/52,592(8LP)
Intel Core i7-3615QM @ 2.3 GHz (4 cores) 1C=623 MC=2154 cf 550/1876
Intel Core i5-3210M @ 2.5 GHz (2 cores) 1C=540 MC=1098 cf 495/1002 cf 2924/11696(4LP)

I started this yesterday and lost the whole thing when I closed too many windows at once, so I am going to send this now and continue in another post. But just for fun I'll add a couple of modern CPUs, to underscore Bikeaddict's argument. https://folding.lar.systems/cpu_ppd/overall_ranks

Modern 2024 CPU - GB6 data s
Intel Core i9-13900KS 3.2 GHz (24 cores) 1C=3105 MC=21817 (32 logical)
Intel Core i9-13900KF 3.0 GHz (24 cores) 1C=2942 MC=19913 (32 logical)
AMD Ryzen 9 7950X 4.5 GHz (16 cores) 1C=2941 MC=19280 (32 logical)

Modern 2024 CPUs - Folding data
#10 Intel Core i9-13900KS 3.2 GHz (24 cores) 1C=29,387 MC=0.94M PPD (n=559 (32LP)
#47 Intel Core i9-13900KF 3.0 GHz (24 cores) 1C=12,027 MC=0.38M PPD (n=4) (32LP)
#06 AMD Ryzen 9 7950X 4.5 GHz (16 cores) 1C=36,999 MC=1.18M PPD (n=7,538) (32LP)

[appepi]

OK. Before moving on, if yous are paying attention you will notice two oddities in the "Modern CPU" data, namely:

Q1: How do you get 32 logical processors out of 24 physical with the i9-13900's? A: Because only 8 have dual threads,
Q2: Why are the PPD's so low for the i9-13900KF versus the Ryzen 9 7950X when their GB6 performance is virtually identical? Hint: look at the numbers of data points. This is a useful warning about taking summary data at face value, including what is here.

OK, Now let me introduce my own elderly devices, which are pretty much at the same age and performance level as the Macs in the previous post, and rated between 93,000 and 133,000 PPD when using all logical processors.

HP Z600, Z800, (2010) Z440 (2014)
Intel Xeon X5690 3.5 GHz (6 cores) x2 1C= 532 MC=3739 cf 3885/93240 (24LP)
Intel Xeon X5660 2.8 GHz (6 cores) x2 1C= 460 MC=3175 cf 5299/127176 (24LP)
Intel Xeon E5-1650 v3 3.5 GHz (6 cores) 1C=1063 MC=5204 cf 11083/132996 (12LP)

I have used data from my own devices because the GB6 Processor benchmarks are for a single Xeon. By way of getting some current data I set Z601 to folding yesterday, for the first time since 2020. It uses 2 x X5660s, but I limited the number of LP's to 12 with the HP Performance Advisor, by denying the folding app access to the second thread on each core. I do this to keep temperatures down (77-80 deg C) with a relatively small performance penalty, because there's only about a 20% addition from the second thread when there's one being fully utilised, as is easily demonstrated if you switch off hyperthreading. As luck would have it I got two WUs with almost identical run-times. They both had 4 day deadlines, and wanted 15.71 versus 15.57 hours of Z601's time. Of course Z601 is a big old box with 48GB of RAM and a Quadro 5000 to keep warm, so it used about 250W with folding paused, and 450 W when Folding in the way described. My current electricity usage rates are (AUD) 64.625 cents/hour at Peak (6hrs 2pm to 8pm), 26.367 cents/hour at off-peak (9 hrs 10pm to 7 AM) and 34.386 cents/hour at other times (9 hrs).

If we now suppose that the base points for a WU reflects its scientific value, and the Quick Return Bonus reflects the added value of finishing sooner rather than later, then I should try to find a continuous block of 15.7 hours to run in, even though this means that 6.7 hours will be at the more expensive "shoulder" rate. So I do this, and 450W per hour x 15.71 hours requires 7.07 KWh, which costs me $2.11. In theory FaH rewards me with 48,575 points for this if you apply the Quick Return Bonus algorithm from https://foldingathome.org/support/faq/points/ with k=0.75, though in fact I only got 40,505 and the Excel solver says you'd get this with k=0.52149609. But then my actual time from firing up Z601 to the return date/time seems to be only 13.84 hours so who knows? The main point is that it would cost me around $52.00 per million points this way. Or, if I stop Z601 after the 9 hours of off-peak, then start up when the next off peak period begins 15 hours later, my elapsed time is now 30.71 hours, so I only get 28,970 points, and though my cost drops to $1.86, this works out at $64.34 per million points.

I have included the full 450W cost of running Z601 because that is in fact what it would cost if I added it to my current folding collection.

By contrast, LAR Systems does its GPU cost estimates for the GPU maximum power consumption alone, ignoring its supporting box, and they apply a rate of (presumably) US$0.10 per KWh, which would be near enough to AU$0.15 per kWh and thus about half my cheapest rate, so their costs per Mpoint are not directly comparable, but they are so extremely different that it hardly matters, as below,

https://folding.lar.systems/gpu_ppd/overall_ranks GPUs
#001 RTX 4090 (AD102) Linux 26.7 MPPD Win 19.8 MPPD (n’s 54,637/292,429) $/MP @ 450W and 10c/KWh = $0.041
#080 RTX 2060 (TU106) Linux 1.8 MPPD Win 1.6 MPPD (n’s=10,703/85,761) $/MP @ 175W and 10c/KWh = $0.221
#114 GTX 1080 (GP104) Linux 1.8 MPPD Win 1.6 MPPD (n’s=6,686/112,647) $/MP @ 180W and 10c/KWh = $0.365
#112 GTX 1070 (GP104) Linux 1.2 MPPD Win 1.6 MPPD (n’s=87,761/121,786) $/MP @ 150W and 10c/KWh = $0.298
#146 GTX 1060 6GB (GP104) Linux 0.80 MPPD Win 0.45 MPPD (n’s=638/133) $/MP @ 125W and 10c/KWh = $0.367

Allowing an additional 250W or so for the box supporting the GPU with its CPU doing very little, it became pretty obvious that there was no reason to waste electricity on CPU folding on any of my devices, and so I invested in some elderly GPUs when prices became reasonable. Relative to my pre-folding careless consumption of electricity, the three RTX2060s in three Z440s routinely grind out 2.5 MPPD during off-peak hours with any necessary extension into that adjacent "shoulder" period, for an incremental cost around $0.32 per MPPD.

In summary, IMHO old CPUs can provide useful support to old GPUs, and the old GPUs work just fine.

[bikeaddict]

Here's a comparison to newer hardware and the typical power:

E5-2690 v4: 400-450K PPD @200W
E5-2697a v4: 500-550K PPD @200W
Ryzen 9 7950X: 1-1.4M PPD @200W (CPU temp limited to 85C in BIOS)

RTX 4060 Ti: 6-7M PPD @110-150W
RTX 4070 Super: 13-14M PPD @170-200W
RTX 4070 Ti: 14-16M PPD @180-200W

[toTOW]

OK. Before moving on, if yous are paying attention you will notice two oddities in the "Modern CPU" data, namely:

Q1: How do you get 32 logical processors out of 24 physical with the i9-13900's? A: Because only 8 have dual threads,
Q2: Why are the PPD's so low for the i9-13900KF versus the Ryzen 9 7950X when their GB6 performance is virtually identical? Hint: look at the numbers of data points. This is a useful warning about taking summary data at face value, including what is here.

A1 : The oddities of big.LITTLE architecture ... 8 performances cores + 8 logical cores on P cores (hyperthreading) + 16 E cores (no hyperthreading on E cores).
A2 : Because big.LITTLE architectures sucks on FAH and require user optimisations ...

If you were planning to upgrade your CPU for FAH, I think you should go with an AMD one as they don't use big.LITTLE architecture yet.

[appepi]

A2 : Because big.LITTLE architectures sucks on FAH and require user optimisations ...

If you were planning to upgrade your CPU for FAH, I think you should go with an AMD one as they don't use big.LITTLE architecture yet.

The LAR-using folders seem to agree with your choice, since there's N=7,538 data points for the Ryzen 9 7950X, but only N=599 for the i9-13900KS which are not so very different. But there's still only N=4 for the i9-13900KF in their database. And both of the i9-13900s have the same architecture, and don't differ much except a small speed hike for the KS over the KF.( see https://www.intel.com/content/www/us/en ... 497,232167) so I would want to see more than N=4 data points. And for the price of either CPU my Z440/RTX2060 combination out-performs them, but mainly, I started programming in the late 1960s, so any 21st century box is amazing from my point of view, and the ones I have will see me out.