How to fully utilize systems?

[zoharek]

Guys,

I would like to fully utilize hardware available to me, so please advise on the details. I have 2 machines to fold on:

1. Workstation:

- single Xeon E5540 @ 2.53GHz with HT
- 6GB 1066MHz DDR2
- 1x Tesla C1060 in PCIEX16 slot
- 1x Tesla C1060 in PCIEX4 slot (can't place it in X16 due to lack of space, Quadro FX doesn't work in X4 slot )
- 1x Quadro FX580 in PCIEX16 slot

On this machine I run 3 GPU2 clients with settings below:
1st Tesla: -advmethods and checked work assignments larger than 10mb
2nd Tesla: -advmethods and checked work assignments larger than 10mb
Quadro: -advmethods and checked work assignments larger than 10m

Plus smp client on Xeon with settings:
-smp -advmethods and bigpackets=big

First thing here I wonder about is what happens if I get exactly the same assignment on 2 GPUs, same project with the same Run, Clone, Gen? Does it count that I process it two times, is it ok? Both teslas tend to get identical assignment from time to time.

Is it better to run them separately or should I somehow make them cooperate?

2. Server:

- Dual Xeon E5520 @ 2.27 GHz with HT
- 16GB 1066Mhz DDR2

On this machine I run smp client with settings:
bigpackets=big extra_parms=-smp -bigadv

I have read something about the bigadv flag, I will see if system is capable of processing these in time allowed.

So basically, I'm good or doing things terribly bad? Not sure why but HFM does not show me PPD, at least not yet.

[Nathan_P]

Welcome to the Forum!!!

Both machines are setup fine, However i would question whether E5520's at stock speeds will meet the preferred deadlines, the only way you will know is by running the client for a while and seeing how you get on. Usually ~55 minutes per frame can be taken as a rough deadline for finishing in time.

Have you got a passkey? you only need one and can install it on multiple machines

If you are getting duplicate Wu's on your clients check that the machineID for each client is different, duplicate WU are sent out but it is very rare, i've folded for 2 years and haven't seen one unless i'm messing with my setup and forget to check machine ID's.

[zoharek]

About E5520, we will see. I have got a passkey generated on F@H webpage.

Machine ID is different, they are set up using FAQ by making different directories in AppData/Roaming. HFM reports them in orange, I guess it's not good right? I now have had 2 consecutive duplicates, I'm fine with that as long as it is not useless for F@H project, don't like to waste workpower.

[uncle fuzzy]

Duplicate work doesn't help anyone. Check the the start of the logs on all your gpus for different machine IDs.

Launch directory: C:\gpu 0
Executable: C:\gpu 0\[email protected]
Arguments: -gpu 0 -verbosity 9

[19:38:24] - Ask before connecting: No
[19:38:24] - User name: uncle_fuzzy (Team 52286)
[19:38:24] - User ID: xxxxxxxxxxxxxxxxx
[19:38:24] - Machine ID: 2

The other 2 GPU clients on this box are 3 and 4.

[zoharek]

Yeah, both teslas were running on the same machine ID, I changed it but it seems client needed restart to use new one. Now I have 4 different jobs, thanks.

Bigadv is too much for the server One percent in 1:40h, makes it do whole job in 7 days, while only 6 days to do it until deadline. Need to organize something more potent :>

[uncle fuzzy]

Somewhere around 57 minutes is the slowest frame time to complete bigadv.

Have you tried the regular SMP, and what were the frame times? My i7 does 25K on bigadv, and 15K on SMP. Not as nice for the totals, but still a good chunk of change.

[zoharek]

Another question, Hyper Threading helps bigadv or is it the opposite?

SMP with advmethods sample:

[19:16:48] Project: 6702 (Run 6, Clone 64, Gen 26)
[19:16:57] Completed 1847260 out of 2000000 steps (92%)
[19:50:49] Completed 1860000 out of 2000000 steps (93%)
[20:28:25] Completed 1880000 out of 2000000 steps (94%)
[20:48:31] Completed 1900000 out of 2000000 steps (95%)
[21:08:43] Completed 1920000 out of 2000000 steps (96%)
[21:27:06] Completed 1940000 out of 2000000 steps (97%)
[21:44:24] Completed 1960000 out of 2000000 steps (98%)
[22:04:33] Completed 1980000 out of 2000000 steps (99%)
[22:24:31] Completed 2000000 out of 2000000 steps (100%)
[22:24:54] Unit 4 finished with 65 percent of time to deadline remaining.

SMP with bigadv sample:

[22:26:52] Project: 2684 (Run 0, Clone 23, Gen 9)
[22:27:14] Completed 0 out of 250000 steps (0%)
[00:14:26] Completed 2500 out of 250000 steps (1%)
[02:04:33] Completed 5000 out of 250000 steps (2%)
[03:52:24] Completed 7500 out of 250000 steps (3%)
[05:41:22] Completed 10000 out of 250000 steps (4%)
[07:30:55] Completed 12500 out of 250000 steps (5%)
[09:20:46] Completed 15000 out of 250000 steps (6%)
[11:02:16] Completed 17500 out of 250000 steps (7%)
[12:43:08] Completed 20000 out of 250000 steps (8%)
[14:25:05] Completed 22500 out of 250000 steps (9%)
[16:05:37] Completed 25000 out of 250000 steps (10%)

This machine is too slow, next one I hope will be much better

[Zagen30]

HT helps bigadv- all the work has to be done at some point, so HT's ability to better utilize downtime in the physical cores works to any SMP work's favor.

[zoharek]

OK, tomorrow I will be setting new machine for bigadv, wish me luck, it's 4x Xeon X7560 with HT plus 64GB 1066MHz RAM, hope it will be enough, because it's most I can get from single machine, next would be clusters only.

Also I wonder about GPU clients, as I previously stated, I run 2x Tesla C1060, each makes about 8000 PPD, however I have read that GTX460 goes as far as 10000 PPD. I could replace Teslas with 2x GTX460 (as long as they need 2x 6pin power connectors). This is weird to me though, I thought Teslas > GTX.

[Zagen30]

it's 4x Xeon X7560 with HT plus 64GB 1066MHz RAM, hope it will be enough, because it's most I can get from single machine, next would be clusters only.

That wasn't a typo, and this is a 32-core system? Please do tell what sort of frame times it gets- this should be ridiculous, in a very good way.

[PantherX]

1 Xeon X7560 = 8 Cores/16 Threads
4 Xeon X7560 = 32 Cores/64 Threads
1 GB RAM for each CPU
64 GB RAM for 64 CPUs (if its a dedicated folding rig, that's too much of RAM)

BTW zoharek, how much did this machine cost you?

My guess is that a bigadv WU will take <24 hours to finish. IIRC, the second fastest time to finish it was 16 hours (Intel, 64 CPUs) while the first was 14 Hours (AMD, 48 CPUs).

Regarding Teslas > GTX, they are but in Medical purposes, CAD etc. F@H doesn't use all those advance features thus a gaming GPU will suffice.

[zoharek]

If I was to buy that machine it would be $38k. We'll see the frametime hopefully tomorrow (if all goes well). And yes, it will be dedicated machine, I didn't want it to have too little RAM, I somewhere read about 0.5G per thread as minimum, and 1G as optimal value.

About Teslas, so it's basically a try and see thing? No one knows which exactly GPU will be best for FAH?

[PantherX]

For Linux/OSX, it was 0.5 GB per Core (it isn't the same as thread) thus the minimum was 4 GB while optimum was 6 GB (0.75 GB/Core). 1 GB per core is more than enough. Right now, bigadv is suspended for Linux (due to stability issues) so on Windows, the RAM Usage is low. It is mostly <2 GB but things may change in the future.

For Fermi the best one for folding/watt/$ is GTX 460. It gives a good amount of PPD when OCed about ~10K PPD. However, as the Client is still in BETA Stage, things may vary once it is finalized.

[zoharek]

This is one thing I can't do with machines I use, I can't overclock, so what I look for is power to (2x6pin card at stock speed) ratio I can have two of those in my workstation, whatever power they use.

[PantherX]

If you mean that you can't OC the CPU then yes, it's a server motherboard thus can't be done. However, the GPUs can be OCed without any difficulty especially as GTX 460 have a lower TDP then GTX 470 & GTX 480. On the other hand, if you personally don't like OC, then I suggest that you get a factory overclocked version.