large SMP WU upload times

[theteofscuba]

Are the results currently being compressed before they are uploaded? It takes about 15 minutes at full bandwidth (~30 KB/s) and it is quite annoying when I have multiple SMP clients consuming all my bandwidth for so long, It really hurts my download speeds.

[PantherX]

Well, I guess you are better off as my ~100 MB WU Result of bigadv uploads at ~10KB/s which takes 2.5 hours. Normal SMP2 WUs take 15 minutes to 1.75 hours depending on the WU Result file size, between 3 MB to 60 MB. I would be happy if they could compress the file sizes so that they would be small.

[toTOW]

Yes they are already compressed ... but your upload speeds are quite slow

[Rum@NoV]

Yes, the uploading of WU's also hurts my internet connection, maybe the V7 client should include a "limit upload speed to xx KB's/s" option?

[kiore]

Well, I guess you are better off as my ~100 MB WU Result of bigadv uploads at ~10KB/s which takes 2.5 hours. Normal SMP2 WUs take 15 minutes to 1.75 hours depending on the WU Result file size, between 3 MB to 60 MB. I would be happy if they could compress the file sizes so that they would be small.

I feel your pain My connection (the fastest available) is minimum 30 minutes and more usually 2 hours for smp2 and for big advanced 2-6 hours each, which is actually better as it only happens once every 2-3 days. For my new proposed build I'm having to factor in a dedicated connection. Here's hoping they can be compressed a little more as they develop.
The variation in time seems to be linked to what time of the day the up/down load occurs and probably how many other people in the neighbourhood are online.

[theteofscuba]

Yes, the uploading of WU's also hurts my internet connection, maybe the V7 client should include a "limit upload speed to xx KB's/s" option?

I love that idea. I also have to insist on downloading a new WU before uploading finished WUs. currently it appears that the cpu sits idle during an upload. If you have to wait 2 hours to finish uploading and download a new wu afterward, that is a lot of down time that is better spent folding.

[7im]

How many frames can your client complete while a WU uploads for 2 hours?

10? 20? 50? How would the fah client determine when to start downloading the next WU? What if the current upload is small, but the next upload is huge? What if the current WU is small and fast, but the next WU is slow and large? Vice Versa?

Do you now see how difficult it would be to make a feature like that work well?

If I had to guess, concurrent downloads and uploads are a good compromise, unless the v7 client programmer is a super genius. Or maybe download the next WU after the current WU completes, then upload the completed WU after the download of the new WU completes. We've seen Tear make that work.

[theteofscuba]

Or maybe download the next WU after the current WU completes, then upload the completed WU after the download of the new WU completes. We've seen Tear make that work.

That is exactly what I intended to suggest.

Download speed is generally not a problem for DSL users, but upload speeds are terrible. It appears that downloading a new WU is not a bandwidth intensive process anyway.

Its usually the SMP clients that have massive upload times. I have not experience any lengthy upload delays with GPU2 (ATI) or with uniprocessor clients.

What is Tear?

[Grandpa_01]

Or maybe download the next WU after the current WU completes, then upload the completed WU after the download of the new WU completes. We've seen Tear make that work.

What is Tear?

Not What Who memberlist.php?mode=viewprofile&u=37

[bruce]

There have been long discussions about minimizing the download/upload time by overlapping whatever can be reasonably overlapped. The final conclusion of those discussions was that the client should should start downloading a new WU immediately when the current WU is finished. The upload can be started at the same time and can be processed concurrently. For short downloads and longer uploads, the upload will continue while the first frames of the new assignment are processed. (As 7im points out, it's not reasonable to start downloading BEFORE the current WU is finished.) In fact, v6 already does this if it's restarted between WUs, but it should do it without the need for a restart.

Whether this feature or something similar makes it into the released version of V7 won't be known until that version is finalized, but it's certainly has been requested a number of times.

[tear]

I'm taking this loop off...

[tear]

In fact, v6 already does this if it's restarted between WUs

No, it does not. It synchronously uploads completed unit(s) _then_ (when successful or after several failed attempts) downloads new unit.

[bollix47]

Having restarted a few times at the end of a WU, while it's uploading or just before it starts the upload, I can confirm that as soon as the client is restarted autosend starts to send and a new WU is downloaded immediately, long before the send completes. The client has usually finished a few frames by the time autosend has completed, at least on the larger uploads.

[PantherX]

Having restarted a few times at the end of a WU, while it's uploading or just before it starts the upload, I can confirm that as soon as the client is restarted autosend starts to send and a new WU is downloaded immediately, long before the send completes. The client has usually finished a few frames by the time autosend has completed, at least on the larger uploads.

I would do this when ever my WU Result size was larger than 30MB. However, I have stopped doing this for the bigadv as the upload time is ~2.5 hours, I give my CPU a "break" and would do some of my CPU intensive work instead.

[tear]

Having restarted a few times at the end of a WU, while it's uploading or just before it starts the upload, I can confirm that as soon as the client is restarted autosend starts to send and a new WU is downloaded immediately, long before the send completes. The client has usually finished a few frames by the time autosend has completed, at least on the larger uploads.

Interesting, care to post the log?

Linux client doesn't return a unit from autosend context until a number of
failures has occurred. Perhaps Windows client does things differently (so
much for a unified codebase)?


tear