Project: 5003 (Run 9, Clone 84, Gen 5) upload problem

[ThunderRd]

Could one of the guys with access please check if this WU has been upped?

p5002_supervillin_e1
P5003 R9 C84 G5

I'm trying to troubleshoot some upload problems I'm having, thanks.


Added WU info to title, in preferred format. -7im

[toTOW]

No record of that WU in the database yet ...

[bruce]

It did show up an hour later and then you uploaded it again twice during the next two hours.

Hi ThunderRd (team 45),
Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 08:36:55 for 479 points of credit.
Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.
Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.

[ThunderRd]

I reckoned that would happen. Still can't figure out why it's happening

For now, since they *seem* to be uploading, but not deleting automatically from the queue, I'm just removing the x.dat file after an appropriate time. Then the client doesn't find the file, and removes it from the queue.

PS I wonder why the times are the same on these entries. Weird.

Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.
Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.

If you see this message twice, everything is ok.
If you see this message twice, everything is ok.

[toTOW]

Do you have some network devices that might alter the ACK messages (ie. proxy, router, transparent ISP proxy, ...)

[uncle_fungus]

If you could please post your log around the upload it would help too.

[ThunderRd]

My story is in this thread:
viewtopic.php?f=43&t=3641&p=35245#p35245

Which was merged into this one:
viewtopic.php?f=18&t=2703&st=0&sk=t&sd=a

So I'm a bit hesitant to open a new discussion here, although the problem isn't solved as yet. I'll just say that I do 20-25k ppd on the smp client on 30 machines (including the one the GPU client is on) with no problem of this kind. The only problem is on the GPU client. If I had to guess, that means it's most likely an issue with the server(s) or the way my ISP sees the server(s). It doesn't seem like a client problem.

[bruce]

PS I wonder why the times are the same on these entries. Weird.

Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.
Your WU (P5003 R9 C84 G5) was added to the stats database on 2008-07-06 10:42:45 for 0 points of credit.

If you see this message twice, everything is ok.
If you see this message twice, everything is ok.

Just a guess, but suppose the WU "failed" to upload to the primary work server and then also "failed" to upload to the collection server. (Two uploads within the same hour will be added to the database at the same time, even if the upload times were different.) In both cases, the WUs would be duplicates of the one that uploaded a couple hours earlier for full credit.

We do need to figure out why the confirmation message is not reaching the client.

[ThunderRd]

Code: Select all

[09:48:05] - Unknown packet returned from server, expected ACK for results

Well, something is reaching the client, but it's not ACK. Is there a way to log it somehow so we can see what the packet is?

[ThunderRd]

Here's something interesting. While tweaking the shader settings on my card, I managed to push too hard, and got an unstable machine error. The client errored, and then uploaded the partial WU. The ACK response must have come, because the log looks like this:

Code: Select all

[05:24:26] Completed 68%
[05:26:26] Completed 69%
[05:27:27] Gromacs cannot continue further.
[05:27:27] Going to send back what have done.
[05:27:28] logfile size: 90742 info=90742 bed=23 hdr=1
[05:27:28] - Writing 91278 bytes of core data to disk...
[05:27:28] Done: 90766 -> 10220 (compressed to 11.2 percent)
[05:27:28]   ... Done.
[05:27:28] 
[05:27:28] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:27:33] CoreStatus = 7A (122)
[05:27:33] Sending work to server
[05:27:33] - Read packet limit of 540015616... Set to 524286976.


[05:27:33] + Attempting to send results
[05:27:33] - Reading file work/wuresults_08.dat from core
[05:27:33]   (Read 10732 bytes from disk)
[05:27:33] Connecting to http://171.64.65.20:8080/
[05:27:40] Posted data.
[05:27:41] Initial: 0000; - Uploaded at ~1 kB/s
[05:27:41] - Averaged speed for that direction ~1 kB/s
[05:27:41] + Results successfully sent
[05:27:41] Thank you for your contribution to Folding@Home.
[05:27:45] Trying to send all finished work units
[05:27:45] - Read packet limit of 540015616... Set to 524286976.

After that, the client properly deleted the x.dat file, and downloaded a new WU. Now, why would it receive an ACK packet on a partially complete WU, and not on a complete one?