The Folding@Home GPU Statistics Database

[compdewd]

Somewhat of an Update:

I have decided to let everyone in on the code that I have been working on that has taken me away from further developing the database page. I have been learning how to use Git for the past few hours too and I must say it's quite cool. Anyway, this side program should have never taken this long to code, but with school and work and (possibly) soon an internship, my time to develop this side program has been taken from me. The purpose of the program is so I can take someone's HFM.NET data and parse it to give an output that corresponds to the information needed for submission to the GPU database. This will certainly ease my job and could potentially ease a submitter's job, allowing them to simply submit a file and not have to worry about filling in several fields.

I have put the code on GitHub and SourceForge, so pick your favorite and check it out (with the links below). Do note, though, that it is not finished, so if you try to compile and run it in it's current form, it will crash. Constructive criticism is more than welcome. I learned C++ (and PHP) on my own (I have yet to take them in college) so things may not look very good to the experienced ones, but in the end the code will get the job done and I enjoy learning from my mistakes and my successes. If you are interested, take a look at the Overview in the README as well. Here are the links to the source code:

The Folding@Home GPU Statistics Database's HFM.NET Custom Parser [... on GitHub] [... on SourceForge]

[compdewd]

Database Update:
I apologize for not posting consistently when new records are added. It's been a while since I have mentioned new records so I'll start again now by saying that there were 6 new records added today.

Ideas for the database:
Being optimistic here about time, but some new ideas for the database are:
- The ability to link to an individual record
- A history of records added (possibly a feed implementation) so this thread does not get cluttered with updates of 1 or 2 new records being added. This would have links to the records themselves.
- The ability to filter records between a pair of dates
- Concatenation of the Chipset and Model columns (only for display)to decrease the width of the table on the web page

[compdewd]

Database Update

• 12 new records were added thanks to bollix47
• 23 errors fixed

EDIT: Didn't mean to give thanks to just bollix47, but rather to all contributors of data to the database. So I should say, 351 records added to the database thanks to you! You are what keeps the database alive!

Sincerely,
Patrick

[compdewd]

Database Update:

• Modified (enhanced) the original post of this thread
• Since I like open source code, I have made available some of the code for the database page on GitHub (more to come soon) which can be found here.
  • Now that the code is available, take note of the LICENSE.txt about modification and redistribution of the code if you're someone that wants to modify and redistribute it. If you are someone that does, please ask me about it. Chances are I'll allow you to, just ask first. Thanks.
  • Note that a code restructure of the entire site, compdewddevelopment.com, is being written and that the code on GitHub has not yet been tested
• More records from bollix47 to appear soon now that I have the parser working for his "WuHistory.db3" file.

[compdewd]

Database Update:

• 62 new records added thanks to bollix47 (because he submitted his HFM.NET's WuHistory.db3 file)

Database Bugs:

• If you happen to bring up the "known database errors" page, you will see an error that says the database is empty, however it is not. Apparently I never had no errors after I created the errors page and could not foresee this bug. This shouldn't be a problem for long, as I'm sure an error will show itself there again soon. It will still be on the "fix list" regardless.
• Speaking of bugs on the errors page, I've just found that the script isn't consistently reporting errors. It's quite strange and may require a rewrite of the script, which by chance is already happening! If you happen to spot an error that doesn't show up on the errors page, be sure to report it!

[compdewd]

CompdewdDevelopment as a whole is down. I am aware of the problem, know how to fix it, and am working to fix it. Apologies for any inconvenience this causes. I will report back when it is back up.

[compdewd]

The site is now back up. Again, sorry for the temporary outage.

[compdewd]

Database Update:

• 26 new records added on March 9, 2013 thanks to zspaldin

[compdewd]

Database Update:

• 18 new records added on March 10, 2013 thanks to PantherX's contribution of two HFM.NET "WuHistory.db3" files

[compdewd]

Database Update:

• 6 new records added thanks to anonymous sources over the past couple days
• 2 new records added thanks to Self (not me, by the way) on March 12, 2013

New Database Plans:

• The database tables are going to be redesigned (along with the code for the site) to have less redundant data. New columns will be added which will be: "Memory Clock Speed" and "Shader Clock Speed". With this table redesign, I foresee being able to add features to the page more easily, and be able to insert records into the database more easily.
• The submission form will change a little. Those submitting will choose from a dropdown list of entities that already exist in the database. There will be an adjacent form that will look similar to the one that exists now where those submitting can add entities that don't exist in the database already (new model, driver version, etc.). Submissions will still go into a secondary table, but this change will allow me to enter them into the main table (database, if you will) more efficiently.

[compdewd]

Database Update:

• 4 new records added over the last few days thanks to anonymous sources
• 1 new record added on March 17, 2013 thanks to Athlonite
• I have thought about the existence of P7662 records (core 0x17 records) in the database and since the core is in beta, I have decided (since it has been the policy in the past) that they should not exist in the database at this time. Submissions of (and currently existing) core 0x17 data will now go into a file that I hold on my personal computer and will be added to the database upon the assignment of core 0x17 to the general public. If P7662's life remains only in the beta community, the P7662 records will be discarded. Oppositions to this policy are welcome (as long as they come with a good reason )

Submission Tips:

• I have noticed that many submissions either lack or have incorrect "FAH Core Version"s. I am creating an FAQ that will eventually have directions to find the core version. For now I will give directions here. The directions are simple:
  1. In your FAH logs, find the beginning of a WU that uses the core that you will specify in your database submission
  2. You should find something like this:

     Code: Select all

     00:47:43:WU01:FS01:FahCore 0x15 started
     00:47:44:WU01:FS01:0x15:
     00:47:44:WU01:FS01:0x15:*------------------------------*
     00:47:44:WU01:FS01:0x15:Folding@Home GPU Core
     00:47:44:WU01:FS01:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
     00:47:44:WU01:FS01:0x15:Build host             AmoebaRemote
     00:47:44:WU01:FS01:0x15:Board Type             NVIDIA/CUDA
     00:47:44:WU01:FS01:0x15:Core                   15

     (Notice the line that specifies the version of the core)
  3. Include the newly discovered core version in your database submission
  Now you can say you know how to find the version of the core used for any WU!

Other Information Related to the Database:

• The busier I become, the more I realize that I may not be able to maintain the database alone. If it gets to a point that (maintenance of) the database suffers due to my priorities in life, I will request one or more assistants whom will assist in entering data into the primary database table in the format that keeps the integrity of the database. It's just being considered at this point. I'm not sure who would want the job, if anyone. If you do want to be a part of the maintenance of the database, just wait until I officially announce that I'm looking for assistance. Thanks

[compdewd]

Database Update
Well, it's around 7:30 in the morning for me here in Cincinnati, Ohio, USA. I have stayed up all night coding the database. I have to go to leave for school in two hours. I think I have finished what I wanted to do for now. Here are the new things:

• The code that runs the database has been revised/simplified. This should make it easier/faster to implement new features (hopefully!). Unfortunately, the user base will not notice a difference between the old code and the new code, only the features that it brings about, so I hope no one thought I was wasting my time rewriting the code that had to be rewritten (it started off as some MySQL queries and I wasn't anticipating the features it would gain).
• An FAQ has been added to the database. Honestly, I really haven't received any questions so for now it's just some questions I thought might be commonly asked. The "About the Database" was merged into the FAQ.
• The ability to filter by more than one criterion at a time has been implemented. Basically, now instead of just filtering by all GPUs of project X, you can filter project X by GPU Y. It may be easier to try it out than for me to try to explain it
• Added the ability to link to a single record, whether in a new page or within a group of records. For example, if you would like to tell someone about a certain record, you can click on the arrows on the left of the record which will open up a page with just that record on it. Copy the URI in the address bar, and you're good to go. Alternatively, you can link to the "fragment identifier" of the record on the page which will keep all the records listed in your current query and redirect those who click on the link to the record in the group of records on the page. In that case, the record being specified will appear at the very top of the screen. The second method is a little harder to do since you must know the statistic record ID (which can be found by clicking on the arrows mentioned in the first method and looking for the "record" number in the URI). I don't know of a reason why someone would choose the second method over the firs (which is why I didn't use my time to make it very easy), but I made it available in case someone needs it.
• The columns have a fixed width in preparation for a scrolling table header. It was a little tricky, but it should be fine now.
• Added a "Shader Clock Speed" and "Memory Clock Speed" columns.
• Removed records that did not have current PSummary projects. These were lost in the transition of the database format and would have been very time consuming to try to salvage. One can look at the database backups if they would like to reference those records that were "lost".

I'm pretty tired so I've probably forgot to mention something and there's probably something wrong with the new code that I've missed debugging, so if you notice anything that fits those categories, give me a holla
You now see the current development progress that matches my end, so it may be a while before anything else comes to be as nothing is "in development". Hopefully not too long, but my schedule is about to get pretty packed coming up in the next few weeks.

As always, thanks for reading!

EDIT: Forgot to upload the new files
EDIT: All better now

[compdewd]

Database Update:

• The form to submit data has been fixed. I had tested it and it worked, but then I changed some code and forgot that that code would change the form code. Long story short, it didn't work, but it's been fixed.

[compdewd]

Personal Update:
I have been quite busy lately. I'm actually writing this while in class. There are some records that need to be added as well as a WuHistory.db3 file that I just received. Unfortunately, to add those records, some intermediate code needs to be written with SQL joins and whatnot. Poor planning on my part, honestly, but if I had time to write the code it would be finished already! Huge apologies for the wait, especially to those who subitted data. I'm excited for that WuHistory.db3 file though

[compdewd]

Database Update:
As many of you probably already know, I have not been active with the database for what feels like a very long time to me. Unfortunately, I have been required to give my attention to things in my life other than the database. I have also not been keeping up with FAH updates so I do not know if maintaining the database in its current form is worth anything to anyone anymore. (I'm thinking FAHBench may have utilized a better way to measure graphics cards performance? I don't know.)

There are some things that are greatly important to those who may still use the database even in its currently outdated form or those who may plan on using it in the future:

• I have hosted compdewddevelopment(.com/.net) with GoDaddy and the domain names and hosting plan are set to expire on December 11, 2013, which is 10 days from the day of this posting. I will renew the domain names so I can keep ownership of them, however, the hosting plan with GoDaddy will cost me $107.88 USD. Because of my financial situation right now, I cannot afford to renew the hosting plan. And seeing as I cannot maintain the database due to my time constraints, I think that the best course of action for me is to drop availability of the resource for an indefinite amount of time. I am reluctant to drop availability since the database page is now the top Google search result for "fah gpu database" and because I believe as much information should be available to the public as possible for the betterment of the humanity. But again, I cannot afford to keep the information available with interactivity. Which brings me to my next point:
• Thanks to the wonderful cloud-storage service provided by Google, I am able to provide a copy of the data in the database (as seen by the user) at no cost to anyone. The file is provided in CSV format for small size and easy interpretation by a spreadsheet-like application. The file can be found with this link: https://drive.google.com/file/d/0B9zKEg ... sp=sharing. The default option for CSV files in Google Drive is to have them downloaded to the user's computer for viewing on the client side. If you do not want to download the file and deal with all the associated implications of dealing with a CSV file, you can click in Google's web app: File -> Open With -> Google Sheets and view it in your browser.

Note:
I do not want to have control of the data provided by the database, nor do I want to seem like I want control. I thought of officially licensing the data under some kind of unrestricted license, but all that trouble seems far too unnecessary. I suppose I can do so if requested, but hopefully people can trust that I am not going to pursue legal action (or any other action) if the data is used elsewhere, and I don't even have the means to do so if I wanted to! So, informally, anyone wishing to use the information supplied is free to do so as long as the information remains truthful

I think I've covered everything I wanted to announce. If you would like to contact me, you may email me at [email protected] or through this great forum.

Thanks for bearing with the unsteady nature of my volunteer work

- Patrick Rebsch