Comparing performance between folding clients

Moderators: Site Moderators, FAHC Science Team

sergd
Posts: 3
Joined: Wed Mar 25, 2009 6:49 am
Location: Seattle, USA

Comparing performance between folding clients

Post by sergd »

Hi all - is there an easy way to compare by how much my GPU client is faster than a CPU client? I keep hearing that it is much faster, but it would be nice to know by how much and, for example, how both of them compare to a PS3 client. From what I see from other posts, points and WU seems to be a client type specific, e.g. NVIDIA GPU points can't accurately compare to ATI GPU points. Is that the case? Is there a way to perform an apple to apple comparison between different clients (e.g. using the same WU compiled for different core and clients)?
Last edited by sergd on Wed Apr 08, 2009 4:29 am, edited 1 time in total.
Zagen30
Posts: 823
Joined: Tue Mar 25, 2008 12:45 am
Hardware configuration: Core i7 3770K @3.5 GHz (not folding), 8 GB DDR3 @2133 MHz, 2xGTX 780 @1215 MHz, Windows 7 Pro 64-bit running 7.3.6 w/ 1xSMP, 2xGPU

4P E5-4650 @3.1 GHz, 64 GB DDR3 @1333MHz, Ubuntu Desktop 13.10 64-bit

Re: Comparting performance between folding clients

Post by Zagen30 »

sergd wrote:Hi all - is there an easy way to compare by how much my GPU client is faster than a CPU client? I keep hearing that it is much faster, but it would be nice to know by how much and, for example, how both of them compare to a PS3 client. From what I see from other posts, points and WU seems to be a client type specific, e.g. NVIDIA GPU points can't accurately compare to ATI GPU points. Is that the case? Is there a way to perform an apple to apple comparison between different clients (e.g. using the same WU compiled for different core and clients)?
This can't happen because the same WUs are not made for multiple clients.

Are you asking about Points Per Day? If you get a monitoring client like FahMon and point it to the client directories, it'll calculate how many points per day each client produces (doesn't work with PS3s). I don't know the site, but there is a webpage that has a database of PPD for most, if not all, of the clients, with multiple hardware configurations. On the ATI and Nvidia forums there are charts of how many PPD most models get. PS3s get approximately 900 PPD when folding 24/7.

The problem with comparing non-PS3 clients is that it depends a good deal on your hardware. The GPU2 client can do as low as a couple hundred PPD (low-end ATI card) to several thousand (Nvidia GTX 280). The SMP client can range from the hundreds (low-end dual-core) to a couple thousand (Core i7). The uniprocesoor client is the only one, other than the PS3, that doesn't have a huge range (only looking at one instance being run), mainly because the current advances in CPU technology seem to be more in number of cores and less in more efficient architecture.
Image
kelliegang
Posts: 90
Joined: Wed Mar 04, 2009 4:30 am
Hardware configuration: L1:Dual Core 1.6ghz, 1GB Ram
L2:
PC1&2:P2 3.2ghz, 1GB Ram, Gforce 6800 :(
PS3
Location: Australia

Re: Comparting performance between folding clients

Post by kelliegang »

Client statistics by OS
The Pande Group from Stanford have done some conversions of the output from each different client type to make them all comparable... how this relates to points per day on each of the clients I've no idea...

If you average it out, each active playstation out there is producing 0.06 [x86]tflops but this is the only client you could possibly hope to come up with that kind of figure for... all the other clients produce a different ppd depending on the system they are being run on.

It'd be nice to have some form of comparitive figure though, would help in determining our personal "ideal" balance for systems which we run.
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Comparting performance between folding clients

Post by 7im »

While the GPU client may be 20x faster at folding than the CPU client, it only earns 10x the points, or whatever the current values are. This is because the work done by the GPU is of a very limited scope, and not 20x as useful as the CPU client. While nowhere near perfect, the best comparison to make at this time is the rate PPD of the benchmark system for each client. 110 CPU, 900 PS3, 1500 GPU, 1760 SMP.

Do NOT compare flops, as those numbers can be deceptive, especially when you take in to account the relative speeds I mentioned above. 20x the flops does not mean the client is 20x faster nor produces 20x the amount of useful science.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
jaak ennuste
Posts: 30
Joined: Thu Jan 08, 2009 11:30 am
Hardware configuration: 3 x Nvidia Geforce GTX 295 cards (total 6 GPU's)
AMD Phenom x4 2600MHz Black Edition
MSI 790FX K9A2 Platinum system board, 4GB RAM
ATX case, 1000 W Chieftec PS
Windows XP 64-bit
More: http://estoniadonates.wordpress.com/
Location: Tallinn, Estonia, EU
Contact:

Re: Comparting performance between folding clients

Post by jaak ennuste »

7im wrote:20x the flops does not mean the client is 20x faster nor produces 20x the amount of useful science.
Are Your sure? Running one 1 GFLOPS PC or running 20 PC-s each 1 GFLOPS. I really think that I give 20x more to science???!

Jaak
building 32 GPU folding rig: 16 x NVIDIA GeForce GTX 295 cards; dual PSU solution; 4 nodes.
Website: Estonia Donates, ambitious 400 PPD supercomputer project
Sponsored by AdlerFlow ceramic coated Porsche exhausts
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Comparting performance between folding clients

Post by 7im »

Again, that is not an accurate assumption if you based it on the GFLOPS.

The CPU client is much slower, but can do a wide variety of calculations. The GPU is much faster, but can only do one type of calculation. Read the history section of the GPU FAQ for more info on the nature of GPU folding.

Note, I am not arguing against you. I am not saying the 20 CPUs are better. I am saying you can't make that judgement based on the GFLOPS. The GFLOPS count is deceptive in the GPU client, as it has to repeat some calculations twice because it can't store the results for some simulations. So while 20x faster, it might only be doing 10x the actual work. I don't have exact numbers, but that should demonstrate why you can't compare GFLOPS.
Last edited by 7im on Sat Apr 04, 2009 5:19 am, edited 1 time in total.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
kelliegang
Posts: 90
Joined: Wed Mar 04, 2009 4:30 am
Hardware configuration: L1:Dual Core 1.6ghz, 1GB Ram
L2:
PC1&2:P2 3.2ghz, 1GB Ram, Gforce 6800 :(
PS3
Location: Australia

Re: Comparting performance between folding clients

Post by kelliegang »

I thought points per day was based purely as a comparison of your system with the benchmark system for that client type and had no real useable inference to the science being achieved.
It would be nice to be able to compare all the different clients together is all, Granny smith apples to luscious red apples at least, rather than apples with oranges ;)
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Comparting performance between folding clients

Post by 7im »

kelliegang wrote:I thought points per day was based purely as a comparison of your system with the benchmark system for that client type and had no real useable inference to the science being achieved.
It would be nice to be able to compare all the different clients together is all, Granny smith apples to luscious red apples at least, rather than apples with oranges ;)
Yes, and then no. You have it half right.

Because the benchmark PPD value is based primarily on scientific production, and since your system performance is evaluated relative to that benchmark, your system PPD is tied to the science it produces. It's not exactly linear, and science isn't the ONLY criteria, but science is the MAIN consideration in regards to the benchmark.

PPD is in the only apples to apples comparison you can make at this time. But again, it's not exact.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
kelliegang
Posts: 90
Joined: Wed Mar 04, 2009 4:30 am
Hardware configuration: L1:Dual Core 1.6ghz, 1GB Ram
L2:
PC1&2:P2 3.2ghz, 1GB Ram, Gforce 6800 :(
PS3
Location: Australia

Re: Comparting performance between folding clients

Post by kelliegang »

Cheers for the clarity.

Though if, in terms of importance, PS3 = CPU = GPU = SMP .... then swouldnt it be a good idea to apply some form of weighting so the ppd be more equal rather than things like 6,000 ppd for nvidia clients compared to 100-150ppd for cpu clients?
Zagen30
Posts: 823
Joined: Tue Mar 25, 2008 12:45 am
Hardware configuration: Core i7 3770K @3.5 GHz (not folding), 8 GB DDR3 @2133 MHz, 2xGTX 780 @1215 MHz, Windows 7 Pro 64-bit running 7.3.6 w/ 1xSMP, 2xGPU

4P E5-4650 @3.1 GHz, 64 GB DDR3 @1333MHz, Ubuntu Desktop 13.10 64-bit

Re: Comparting performance between folding clients

Post by Zagen30 »

kelliegang wrote:Cheers for the clarity.

Though if, in terms of importance, PS3 = CPU = GPU = SMP .... then swouldnt it be a good idea to apply some form of weighting so the ppd be more equal rather than things like 6,000 ppd for nvidia clients compared to 100-150ppd for cpu clients?
I think the issue is that it's kind of hard to rate the clients in terms of importance because they each have their advantages and weaknesses, and it's kind of hard to average out the benefits to some sort of hardline figure.

Maybe the only ones that could be compared are SMP vs. CPU since they both run on the same hardware. I think I once saw that part of the reason why the SMP client gets more PPD vs. running as many instances of the uniprocessor client as you have physical cores is that the SMP has the advantage of inter-core communication, allowing more science to be done (the other parts are that the SMP clients are still not nearly as stable as the CPU client and have much shorter deadlines).
Image
jaak ennuste
Posts: 30
Joined: Thu Jan 08, 2009 11:30 am
Hardware configuration: 3 x Nvidia Geforce GTX 295 cards (total 6 GPU's)
AMD Phenom x4 2600MHz Black Edition
MSI 790FX K9A2 Platinum system board, 4GB RAM
ATX case, 1000 W Chieftec PS
Windows XP 64-bit
More: http://estoniadonates.wordpress.com/
Location: Tallinn, Estonia, EU
Contact:

Re: Comparting performance between folding clients

Post by jaak ennuste »

Isn't there now apples-to-apples comparisin x86 FLOPS metrics up and running now?
http://fah-web.stanford.edu/cgi-bin/mai ... pe=osstats
building 32 GPU folding rig: 16 x NVIDIA GeForce GTX 295 cards; dual PSU solution; 4 nodes.
Website: Estonia Donates, ambitious 400 PPD supercomputer project
Sponsored by AdlerFlow ceramic coated Porsche exhausts
PlayLoud
Posts: 61
Joined: Mon Dec 10, 2007 8:29 pm

Re: Comparting performance between folding clients

Post by PlayLoud »

Not if what 7im said is true, in that the GPU clients have to rerun some of those calculations, because they do not retain the results in memory like a CPU would.
Folding@home: 24/7
- RTX 4070 Ti

Image
kelliegang
Posts: 90
Joined: Wed Mar 04, 2009 4:30 am
Hardware configuration: L1:Dual Core 1.6ghz, 1GB Ram
L2:
PC1&2:P2 3.2ghz, 1GB Ram, Gforce 6800 :(
PS3
Location: Australia

Re: Comparting performance between folding clients

Post by kelliegang »

Well Jaak, if I understand what is being said, the argument is that flops dont represent the "science" as clearly as points per day does I think? Surely though you could easily give an x86 factor instead of points per day [figure out total flops produced by one client type, divide this by the points contributed by that client type; voila x86 rating per point]
Pick2
Posts: 85
Joined: Fri Feb 13, 2009 12:38 pm
Hardware configuration: Linux & CPUs
Location: USA

Re: Comparting performance between folding clients

Post by Pick2 »

If the only thing the clients did were "flops" , It might be easier to figure. The clients are calculating the path each atom takes as it morphs from one compound into another. It is much more complex than "How many times can I multiply each second".
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Comparting performance between folding clients

Post by bruce »

kelliegang wrote:Though if, in terms of importance, PS3 = CPU = GPU = SMP .... then swouldnt it be a good idea to apply some form of weighting so the ppd be more equal rather than things like 6,000 ppd for nvidia clients compared to 100-150ppd for cpu clients?
I don't think that you can say that PS3 = CPU = GPU = SMP. In fact, all are important but it is very difficult to know how to compare them. To make a Waldorf salad, you need a mixture of fruits. If it's all apples or all oranges or all nuts, it's not a good salad. If you have two salads with different mixtures, both can be acceptable.

If there are reasonable variations in the production of the four clients, science will get done. If too many people choose NOT to run, say the SMP client becuase it's harder to make work, then the Pande Group might consider giving it more points to influence your choices, but the generally tend to make no changes as long as the overall mixture is in the reasonable range.
Post Reply