It's getting better now, isn't it?
Moderators: Site Moderators, FAHC Science Team
It's getting better now, isn't it?
Is it just me or are the servers becoming smoother, at last? I'm reaching some 50% of my Total Estimated Points Per Day! And most of the time all my ressources are now used.
The servers stats are now constantly over 100,000 jobs per hour: https://apps.foldingathome.org/serverstats - kudos to LTT (I suppose) for their linus1 server!
Feels great!
The servers stats are now constantly over 100,000 jobs per hour: https://apps.foldingathome.org/serverstats - kudos to LTT (I suppose) for their linus1 server!
Feels great!
-
- Posts: 2522
- Joined: Mon Feb 16, 2009 4:12 am
- Location: Greenwood MS USA
Re: It's getting better now, isn't it?
I was up to 1/2 expected PPD, late last week, back to 1/4 now. When researchers get back Monday I am optimistic more resources will get online.
As to how well they are doing, look at this graph, click on Weekly and Users Active
https://folding.extremeoverclocking.com ... ary.php?s=
That is impressive!
As to how well they are doing, look at this graph, click on Weekly and Users Active
https://folding.extremeoverclocking.com ... ary.php?s=
That is impressive!
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
Re: It's getting better now, isn't it?
It seems to be better here, although I do have to reboot now and then to get a work unit or two.
But, may I ask, are the scientists learning anything from the huge increase in computing power, and how long
will it take to get the information into the hands of the researchers that need the information? It'd be nice to
know how forward we're moving in solving this. Is F@H regular work also progressing faster? Thanks!
But, may I ask, are the scientists learning anything from the huge increase in computing power, and how long
will it take to get the information into the hands of the researchers that need the information? It'd be nice to
know how forward we're moving in solving this. Is F@H regular work also progressing faster? Thanks!
Reality is what you stumble over when you walk around with your eyes closed.
-
- Posts: 1996
- Joined: Sun Mar 22, 2020 5:52 pm
- Hardware configuration: 1: 2x Xeon [email protected], 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon [email protected], 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: [email protected], 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21 - Location: UK
Re: It's getting better now, isn't it?
I expect we will get some form of News announcement as/when the team have a chance to do one - but I expect resource is focused at resolving the server throughput challenges ... I would expect the knowledge is already filtering out to the researchers as this is I believe shaping the directions and scale of the new projects - which are spawning fast and furious at the moment (keeping the volunteer beta testers well busy).
Given the current pandemic work is being given priority in the queues and as I understanding the issues are not lack of WUs but more lack of ability to farm out quick enough to keep all the expanded community busy I believe that for the most part it is Covid related WUs being released ... Some regular work may well be getting out at the moment and once expansion of
throughput on the servers is resolved then I expect that as long as regular science WUs are being generated they will get folded.
(Brain wanders off at a tangent)
I wish it were possible to "solve" Covid ... I am simply hoping that all the myriad efforts inc FAH, and lifestyle changes, and shifts in societal behaviours can get us to the point where a new normal can be established that doesn't require vast groups of people to be isolated and where enough has been learned that ways of living develop and sustain that addresses some of the deep seated causes of the pandemic (and I don't mean the virus itself).
Given the current pandemic work is being given priority in the queues and as I understanding the issues are not lack of WUs but more lack of ability to farm out quick enough to keep all the expanded community busy I believe that for the most part it is Covid related WUs being released ... Some regular work may well be getting out at the moment and once expansion of
throughput on the servers is resolved then I expect that as long as regular science WUs are being generated they will get folded.
(Brain wanders off at a tangent)
I wish it were possible to "solve" Covid ... I am simply hoping that all the myriad efforts inc FAH, and lifestyle changes, and shifts in societal behaviours can get us to the point where a new normal can be established that doesn't require vast groups of people to be isolated and where enough has been learned that ways of living develop and sustain that addresses some of the deep seated causes of the pandemic (and I don't mean the virus itself).
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
-
- Posts: 71
- Joined: Sun Dec 02, 2007 4:29 pm
- Hardware configuration: Gigabyte Aorus Z590 Pro AX, Intel i9-10850K, 32GB Crucial Ballistix DDR4-2600, Samsung NVMe EVO 980 Pro 256GB, CoolerMaster liquid cooler ML360, Nvidia Titan X (Pascal), Dell Nvidia RTX 3080 10GB 4Y12V, Pop!_OS.
- Location: Fair Play, SC
Re: It's getting better now, isn't it?
I am getting plenty of work, but my backlog of uploading jobs is twenty at this time. FAH has used more than 20GB of bandwidth in the past few days trying to upload work units. Most fail repeatedly at 40% to 99% uploaded. If the servers would limit their number of simultaneous connections, the throughput would improve.
-
- Posts: 78
- Joined: Wed Mar 25, 2020 2:39 am
- Location: Canada
Re: It's getting better now, isn't it?
Possibly. But I imagine that to some extent, it's just the sheer volume of incoming requests. It's like a friendly DDoS attack. Even if the server were configured to ignore half of the requests directed at it, all of those incoming packets are still filling up the "pipe".Vester wrote:I am getting plenty of work, but my backlog of uploading jobs is twenty at this time. FAH has used more than 20GB of bandwidth in the past few days trying to upload work units. Most fail repeatedly at 40% to 99% uploaded. If the servers would limit their number of simultaneous connections, the throughput would improve.
I have never been involved with operating a server that deals with any significant amount of traffic, but I would be surprised if mitigating something like this were as simple as limiting the number of connections.
-
- Posts: 1996
- Joined: Sun Mar 22, 2020 5:52 pm
- Hardware configuration: 1: 2x Xeon [email protected], 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon [email protected], 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: [email protected], 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21 - Location: UK
Re: It's getting better now, isn't it?
It iisn't !!
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
Re: It's getting better now, isn't it?
There are periods of server-overload and gaps where work can be downloaded or uploaded and I think the gaps are becoming more plentiful.
It's still a supply_and_demand issue. FAH has dramatically increased supply and concurrently I suppose demand has backed off. It's hard to measure the relative significance of either one based on looking at the status of my clients a few times a day and I don't closely monitor my points like ajm did in his second post.
My impression is that downloading has improved somewhat but reduced congestion there (if that's what happened) has allowed us to see congestion on back-end systems ... like the points DB.
It's still a supply_and_demand issue. FAH has dramatically increased supply and concurrently I suppose demand has backed off. It's hard to measure the relative significance of either one based on looking at the status of my clients a few times a day and I don't closely monitor my points like ajm did in his second post.
When the bandwidth limiters kick in, you get the message "No WUs available for this configuration" but you get exactly the same message when there's something wrong with your configuration or the servers really have run out of a class of WUs.I would be surprised if mitigating something like this were as simple as limiting the number of connections.
My impression is that downloading has improved somewhat but reduced congestion there (if that's what happened) has allowed us to see congestion on back-end systems ... like the points DB.
Posting FAH's log:
How to provide enough info to get helpful support.
How to provide enough info to get helpful support.
Re: It's getting better now, isn't it?
Hum, my second post doesn't show my points. It's the Aggregate Teams Summary, with a view of FAH's total daily points in the course of last month.
EDIT: And it shows - that was the point - that the upwards progression has been accelerating these last days.
EDIT: And it shows - that was the point - that the upwards progression has been accelerating these last days.
Re: It's getting better now, isn't it?
It's been a lot better within the past two to three days compared to last week.
-
- Posts: 14
- Joined: Wed Jun 18, 2008 10:42 am
- Hardware configuration: Q9550 @ 2.8 GHz
WIN10 x64
2x Radeon R9 280X-3GB
1x Radeon R9 7950-3GB - Location: Amsterdam, The Netherlands
Re: It's getting better now, isn't it?
It's getting better in the sense of getting work, less idle time, BUT I do see a problem with uploading the results ("Server responded PLEASE_WAIT") which is not good for the PPD as well.
Although it is likely worse if the results are not getting uploaded at all (in time)...
Although it is likely worse if the results are not getting uploaded at all (in time)...
Q9550 @ 2.8 GHz | 2x R9 280X-3GB | HD 7950-3GB | Win10 x64
Re: It's getting better now, isn't it?
Browsing my logs, I see that the upload to 13.90.152.57 was often slow to very slow lately.
37.187.12.48 was fast.
13.82.98.119 was fast.
140.163.4.231 was ok.
Is that kind of feedback useful?
37.187.12.48 was fast.
13.82.98.119 was fast.
140.163.4.231 was ok.
Is that kind of feedback useful?
Re: It's getting better now, isn't it?
Help!!! F@H is getting spoofed badly. Here are the details: <https://www.proofpoint.com/us/threat-in ... l-campaign>
Reality is what you stumble over when you walk around with your eyes closed.
Re: It's getting better now, isn't it?
Well, there's not much FAH can do. And, don't worry, if you're here, you have chosen the real one.