FCI: Folding@Home Client Info (Web based client monitor)

[Shadowtester]

Code: Select all

NAS:/usr/local/bin# ./fci-update-eoc-stats.pl --verbose
Can't locate RRDs.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.8.8                  /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /                 usr/share/perl/5.8 /usr/local/lib/site_perl .) at ./fci-update-eoc-stats.pl line                  90.
BEGIN failed--compilation aborted at ./fci-update-eoc-stats.pl line 90.
NAS:/usr/local/bin#

Here you go I will try to reinstall the librrds-perl package to see it that helps shortly

Ok I installed the librrds-perl package and ran the update-eoc-stats script using verbose I do have a graph now but no data on it
I would post the data but it over ran the putty terminal history I can only get part of the output but it said the script completed here is the output I could get from putty.

Code: Select all

        --vertical-label "Overall Rank"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Overall_Rank=eoc-data/Shadowtester-10688.rrd:Overall_Rank:AVERAGE
        LINE1:Overall_Rank#0000FF:Overall Rank
        GPRINT:Overall_Rank:AVERAGE:Avg\: %.2lf %s
        GPRINT:Overall_Rank:MIN:Min\: %.2lf %s
        GPRINT:Overall_Rank:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-Overall_Rank-1year.png
        --start -8784hours
        --end 1240601116
        --title "Shadowtester (10688) - Overall Rank"
        --vertical-label "Overall Rank"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Overall_Rank=eoc-data/Shadowtester-10688.rrd:Overall_Rank:AVERAGE
        LINE1:Overall_Rank#0000FF:Overall Rank
        GPRINT:Overall_Rank:AVERAGE:Avg\: %.2lf %s
        GPRINT:Overall_Rank:MIN:Min\: %.2lf %s
        GPRINT:Overall_Rank:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points_24hr_Avg-1day.png
        --start -24hours
        --end 1240601116
        --title "Shadowtester (10688) - Points 24hr Avg"
        --vertical-label "Points 24hr Avg"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points_24hr_Avg=eoc-data/Shadowtester-10688.rrd:Points_24hr_Avg:AVERAGE
        LINE1:Points_24hr_Avg#0000FF:Points 24hr Avg
        GPRINT:Points_24hr_Avg:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points_24hr_Avg:MIN:Min\: %.2lf %s
        GPRINT:Points_24hr_Avg:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x268
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points_24hr_Avg-1week.png
        --start -168hours
        --end 1240601116
        --title "Shadowtester (10688) - Points 24hr Avg"
        --vertical-label "Points 24hr Avg"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points_24hr_Avg=eoc-data/Shadowtester-10688.rrd:Points_24hr_Avg:AVERAGE
        LINE1:Points_24hr_Avg#0000FF:Points 24hr Avg
        GPRINT:Points_24hr_Avg:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points_24hr_Avg:MIN:Min\: %.2lf %s
        GPRINT:Points_24hr_Avg:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x268
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points_24hr_Avg-1month.png
        --start -744hours
        --end 1240601116
        --title "Shadowtester (10688) - Points 24hr Avg"
        --vertical-label "Points 24hr Avg"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points_24hr_Avg=eoc-data/Shadowtester-10688.rrd:Points_24hr_Avg:AVERAGE
        LINE1:Points_24hr_Avg#0000FF:Points 24hr Avg
        GPRINT:Points_24hr_Avg:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points_24hr_Avg:MIN:Min\: %.2lf %s
        GPRINT:Points_24hr_Avg:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x268
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points_24hr_Avg-1year.png
        --start -8784hours
        --end 1240601116
        --title "Shadowtester (10688) - Points 24hr Avg"
        --vertical-label "Points 24hr Avg"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points_24hr_Avg=eoc-data/Shadowtester-10688.rrd:Points_24hr_Avg:AVERAGE
        LINE1:Points_24hr_Avg#0000FF:Points 24hr Avg
        GPRINT:Points_24hr_Avg:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points_24hr_Avg:MIN:Min\: %.2lf %s
        GPRINT:Points_24hr_Avg:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x268
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points-1day.png
        --start -24hours
        --end 1240601116
        --title "Shadowtester (10688) - Points"
        --vertical-label "Points"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points=eoc-data/Shadowtester-10688.rrd:Points:AVERAGE
        LINE1:Points#0000FF:Points
        GPRINT:Points:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points:MIN:Min\: %.2lf %s
        GPRINT:Points:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points-1week.png
        --start -168hours
        --end 1240601116
        --title "Shadowtester (10688) - Points"
        --vertical-label "Points"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points=eoc-data/Shadowtester-10688.rrd:Points:AVERAGE
        LINE1:Points#0000FF:Points
        GPRINT:Points:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points:MIN:Min\: %.2lf %s
        GPRINT:Points:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points-1month.png
        --start -744hours
        --end 1240601116
        --title "Shadowtester (10688) - Points"
        --vertical-label "Points"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points=eoc-data/Shadowtester-10688.rrd:Points:AVERAGE
        LINE1:Points#0000FF:Points
        GPRINT:Points:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points:MIN:Min\: %.2lf %s
        GPRINT:Points:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-Points-1year.png
        --start -8784hours
        --end 1240601116
        --title "Shadowtester (10688) - Points"
        --vertical-label "Points"
        --width 550
        --height 175
        --imgformat PNG
        DEF:Points=eoc-data/Shadowtester-10688.rrd:Points:AVERAGE
        LINE1:Points#0000FF:Points
        GPRINT:Points:AVERAGE:Avg\: %.2lf %s
        GPRINT:Points:MIN:Min\: %.2lf %s
        GPRINT:Points:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-WUs-1day.png
        --start -24hours
        --end 1240601116
        --title "Shadowtester (10688) - WUs"
        --vertical-label "WUs"
        --width 550
        --height 175
        --imgformat PNG
        DEF:WUs=eoc-data/Shadowtester-10688.rrd:WUs:AVERAGE
        LINE1:WUs#0000FF:WUs
        GPRINT:WUs:AVERAGE:Avg\: %.2lf %s
        GPRINT:WUs:MIN:Min\: %.2lf %s
        GPRINT:WUs:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-WUs-1week.png
        --start -168hours
        --end 1240601116
        --title "Shadowtester (10688) - WUs"
        --vertical-label "WUs"
        --width 550
        --height 175
        --imgformat PNG
        DEF:WUs=eoc-data/Shadowtester-10688.rrd:WUs:AVERAGE
        LINE1:WUs#0000FF:WUs
        GPRINT:WUs:AVERAGE:Avg\: %.2lf %s
        GPRINT:WUs:MIN:Min\: %.2lf %s
        GPRINT:WUs:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-WUs-1month.png
        --start -744hours
        --end 1240601116
        --title "Shadowtester (10688) - WUs"
        --vertical-label "WUs"
        --width 550
        --height 175
        --imgformat PNG
        DEF:WUs=eoc-data/Shadowtester-10688.rrd:WUs:AVERAGE
        LINE1:WUs#0000FF:WUs
        GPRINT:WUs:AVERAGE:Avg\: %.2lf %s
        GPRINT:WUs:MIN:Min\: %.2lf %s
        GPRINT:WUs:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:
rrdgraph eoc-data/Shadowtester-10688-WUs-1year.png
        --start -8784hours
        --end 1240601116
        --title "Shadowtester (10688) - WUs"
        --vertical-label "WUs"
        --width 550
        --height 175
        --imgformat PNG
        DEF:WUs=eoc-data/Shadowtester-10688.rrd:WUs:AVERAGE
        LINE1:WUs#0000FF:WUs
        GPRINT:WUs:AVERAGE:Avg\: %.2lf %s
        GPRINT:WUs:MIN:Min\: %.2lf %s
        GPRINT:WUs:MAX:Max\: %.2lf %s
        COMMENT:Last Update\: 2009-04-24 14\:25\:16
Image Size: 647x254
Averages:

Saving eoc-stats XML file ...
Successfully saved eoc-stats XML file
NAS:/usr/local/bin#

[smoking2000]

Right, verbose in this case really means verbose, so `fci-update-eoc-stats.pl -v 2>&1 | less` is your friend (assuming you have less installed, it's the better more)

Since all the graphs show nan (Not A Number) in the legend (which you can also see by running `rrdtool dump Shadowtester-10688.rrd | less`, this probably means the rrdupdate fails, or contains invalid data. The next update should show the first data point in the graph.

[Shadowtester]

Well it seems eoc stats are working now and creating the graphs even great work thanks I did not even know that there was a problem.

[smoking2000]

My ISP will assign me a new IP in the near future, this will effect all services hosted behind the following hostnames:
- fci.bile.nl (FCI project website, alternative fci.linuxminded.nl will likely have its DNS changed fastest)
- fci.fatalerrorgroup.com (FCI server for my subteam used as the FCI demo site, this hostname has no alternative)
- linuxminded.nl (My website hosting qd, qfix, etc, alternative linuxminded.xs4all.nl should be updated automatically)

[Shadowtester]

Thanks for the heads up and I hope your recovery is coming along good.

[WangFeiHong]

Sorry i'm really noob at server stuff, is it possible to use free web hosts that can support scripts like PHP and what not? if so, how could i "install" the server components on it?

Thanks...

[smoking2000]

To install FCI you need apache with mod_perl support, simple webhosting usually only supports PHP & CGI. That won't do for FCI. You also need to install a bunch of Perl modules, which requires administrative access to your (virtual) server.

I cannot recommend running an FCI server if you're not comfortable administering a Linux server or have someone do this for you.

[Shadowtester]

As for running it on a Linux server if you do not have a linux box you can use one alternative that I used in the past was a NAS box running debian linux I used my Linkstation Pro 500gb NAS running freelink to run FCI for a while which worked fine but you need some linux skills to do that. One advantage is that its just a NAS which if empty is easy to reflash with the OS and start over if your mess it up real bad. I am not sure how well the Linkstation would be able to handle the latest versions of FCI with all the new graphs and such as its pretty limited with processor power and memory.

[smoking2000]

It appears that my DSL connection is down. I'm currently at work so I cannot troubleshoot the connection at the moment, so the websites I host will be down until I get home tonight.

[smoking2000]

It appears that my DSL connection is down. I'm currently at work so I cannot troubleshoot the connection at the moment, so the websites I host will be down until I get home tonight.

Small update: I just spoke with my ISP, and today they planned to upgrade my connection, so this explains the outage.

I'll you posted on the progress, and/or restoration of services notification.

[smoking2000]

I just got back, fix the NAT table, and we're back in business

[rhavern]

After running smoothly for weeks/months (thanks to your help), I've gotten the following error on my FCI server:
Error running menu.plc:

Ran out of memory for input buffer at /usr/lib/perl5/XML/Parser/Expat.pm line 469

Any ideas where I should be looking to fix this? I haven't touched the server since the last FCI software upgrade cycle.

The server itself is only running at 50% memory load and only FCI runs on this VM. Let me know if you need any information.

Many thanks.

[smoking2000]

Ran out of memory for input buffer at /usr/lib/perl5/XML/Parser/Expat.pm line 469

Based on the error message I'd say that the XML parser runs out of RAM to use when parsing the XML files behind the menu.plc.

The fact that your server is running at 50% memory usage during normal operation is not unexpected, you should have a look at your memory usage when fci-update-xml-files.pl is running, this is one of the most memory intensive scripts, but it releases the memory it uses after each run. The mod_perl scripts powering the FCI website don't release the memory they use after each run, this memory usage is managed by apache mod_perl. It won't hurt to restart your apache to see if this releases some memory, enough to run the website normally again. But you apache webserver should be restarted every week when the logs are rotated, so restarting apache won't necessarily fix this memory issue.

The menu.plc page parses the following XML files:
* xml-data/team-list.xml
* xml-data/client-list.xml
* xml-data/known-projects.xml
* xml-data/client-projects.xml
* xml-data/queue-graphs.xml
* xml-data/fci-summary.xml

Please check the filesize of these XML files, as one or more may have grown too large (for the amount of RAM you have available). It may also be helpful in determining which file is cause the memory issue by commenting out all but one of the read_xml() invocations at the top of the menu.plc file and replace the call with an empty hash reference. If only one of the files is causing this problem you should find which one it by having menu.plc parse just one at a time.

Start out with only the team-list.xml which would make the section of code in menu.plc look like:

Code: Select all

        my $team_list       = read_xml(%args, Name => 'team-list')
                                        || die "Failed to read team XML file!\n";
#        my $client_list     = read_xml(%args, Name => 'client-list')
#                                        || die "Failed to read client-list XML file!\n";
         my $client_list = {};
#        my $known_projects  = read_xml(%args, Name => 'known-projects')
#                                        || die "Failed to read known-projects XML file!\n";
         my $known_projects = {};
#        my $client_projects = read_xml(%args, Name => 'client-projects')
#                                        || die "Failed to read client-projects XML file!\n";
         my $client_projects = {};
#        my $queue_graphs    = read_xml(%args, Name => 'queue-graphs')
#                                        || die "Failed to read queue-graphs XML file!\n";
         my $queue_graphs = {};
#        my $fci_summary     = read_xml(%args, Name => 'fci-summary')
#                                        || die "Failed to read fci-summary XML file!\n";
         my $fci_summary = {};

If all read_xml() invocations are successful when only 1 XML file is parsed by menu.plc, the memory error is likely caused by the accumulated size of the parsed XML files. There may be enough RAM for one or several XML files, but not all of them.

While I was typing this I also did a Google search for the error message which turned up a Debian Bug report for libxml-parser-perl which is used by FCI via the XML::Simple module (which in turn is used by the read_xml() subroutine):
#498194 - libxml-parser-perl: script crashes when reading empty file in xml::rss

The bug report shows that Expat will choke with this same memory error when it's parsing an empty file. This is a known issue of FCI too (because it uses XML::Simple). This should be checked first, to see if any of the XML files used by menu.plc are empty. If they are, replace the XML file with the one in the FCI tarball, which is not-empty (it contains an XML structure) but does not contain any specific information yet. The files will be regenerated by fci-update-xml-files.pl and friends.

Also make sure you don't have any permission problems that may cause the FCI scripts to be unable to parse some files causing the resulting XML files to be empty.

[smoking2000]

It looks like you managed to fix the issue, what did you do to fix it?

[rhavern]

It turned out to be the empty file issue. Many thanks for in pointing me in the right direction!

R