Standardizing the Core/Client to Third-Party Interface

[jcoffland]

We have talked about creating a completely open API for talking to the new API. I would like to use a socket interface because I want to be able to access clients remotely. However, I could also supply a library interface which implements the socket client and exposes an API. Third-party apps could link to this library to gain access to the client's information with out writing socket code.

Honestly, the client API is already defined in a software specification. However, think you will be happy with it. It gives you all the access you want and more and is easy to use both manually and programatically.

I am all for fully opening and publicly documenting these interfaces. However, I will need to double check with the bosses before I can promise anything, but it looks positive.

[bruce]

Honestly, the client API is already defined in a software specification. However, think you will be happy with it. It gives you all the access you want and more and is easy to use both manually and programatically.

I suspect that you're talking about the information that is provided here: viewforum.php?f=41. As far as I know, it has been pretty much ignored by the 3rd party developers.

[kasson]

No, this is a completely new and unreleased client with a new API. I think we'll need to have some careful internal discussions about this. It would be nice to make things simpler for third-party developers, but we also have a difficult line to walk.

[rhavern]

This is another vote for FCI (smoking2000 is my programming hero) and one for fahspy.

Also, another +1 for involving the community PRIOR to making changes. I'm feeling the love!

[EvilAlchemist]

No, this is a completely new and unreleased client with a new API. I think we'll need to have some careful internal discussions about this. It would be nice to make things simpler for third-party developers, but we also have a difficult line to walk.

I can not speak for others, but this is of top priority to me.

Those users with large number of Folding@Home systems MUST have a way to view the status of the clients at a glance.

If the Pande Group can't/won't release the information to 3rd party developers on how to access the clients (to gather WU data), then the Pande Group needs to release a monitoring tool for the community to use.

Checking each system / client's status physically is just not an option when dealing with large groups of computers.

[tear]

Just an opinion here (I don't work that close with data files/logs) --

ad socket interface
Yes, absolutely. I think smoking2000's suggestions (there) re retrievable data are very valuable.

I think there should be only one source of data for third-party applications.
Having two interfaces (separate for core and separate for client) could be
a bit of an overkill for developers.

I'd suggest client becomes abovementioned "source". If client needs to retrieve
(or receive) data from core -- it should do it behind the curtain (that interface
wouldn't *need* to be public).

What else... Yes, I'm wondering whether it would make sense to have both
"STATUS" PDU (so interface client can send query and check what's cooking)
and "EVENT" PDU (so on-line monitoring apps don't need to continuously poll
for changes).

Interface should be extensible too so tools have a chance of working when its
new spin comes out.

A client library is a very good idea if there 's enough resources to create it; I'm pretty
sure community would create bindings for other tools/languages.



I'd have one suggestion for logging though; there are tools that rely on information
from the log (timestamp + WU progress, for instance). If we exclusively stick to
socket interface such tools would need to track progress of all clients (in
an on-line manner) -- I don't think it's feasible.

Thus, I'm suggesting relevant log messages are prefixed with some magic so tools
can "tune" to it and tool maintainers don't need to worry about any kind of extra messages
that may be added by core developers.

It shouldn't be much of a hassle if client<->core interface is in place.



tear

[calxalot]

It would be great if the client advertises the port via Bonjour/Avahi/ZeroConf.
A restful http service might also be nice. Or even just a status page people could see from their web browser.

[MtM]

What hasn't been mentioned yet, and maybe because there is hope for a central 'control application' for all local clients, is that not only status queries and events would be usefull but also an external api to control/configure clients. Configuring is possible externally using .net streamwriter/readers and process.standardout and standardin but it's cumbersome, controlling is only possible by sending a wm_close command which works but I would like more options like pause to be issued externally. For instance one could then implement a 'pause all clients' when a preconfigured kind of application is detected ( like a game or an otherwise compute intensive application ).

If a central monitor and control center from Stanford is in the pipeline this offcourse can be ignored.

If not, enabling this type of control will enable 3rd party developers to take that burden on them, I treid in the past but I wasn't able to do everything I wanted, and the things in which I succeeded like making a gui for configuring console clients are not efficient ( for a single client, I have to start -configonly 2 times, one for reading standard settings, one for reading advanced settings ( due to diffrent configuration options in gpu/smp clients it can not be done in one read ) and this can take up to 15 seconds just filling in the gui with current settings. Applying them needs to start the process, write values to standardout, and then read all settings back with again two reads to verify.

[codysluder]

Pardon my ignorance, but would a .net streamwriter application be used on Linux, OS-X, the PS3 or would it be just for Windows?

. . . and the first question the Pande Group is going to ask is how will this improve the science, or does it represent an increase in production of 0.000000001% (sarcastic wink).

[MtM]

Yes afaik mono supports the entire system.diagnostics namespace.

I made posts with links to source from maxFah which uses this in the past, I don't think my server/webspace is still running though ( and I should remove the reference to it as with current conditions I don't think it will give a proper end user experience ).

The issue is, it's cumbersome, and error prone. One in each x number of process starts result in the client being 'hung', not outputting the expected data, and I had to use a timer based check to restart the current operation ( reading in settings, writing settings through standardinput ).

So it is already possible to build a shell for configuring console's hidden from view, but the manner to do so is not optimal not even close.

If you're interested in the code send me a pm I'll send you a link or an email with the class I made for that purpose, but I think it's rather pointless to look into it further unless we get an improved interface.

And to answer that first question: you been around long enough, what has been f@h's only ( well most complained about ) shortcomming compared to BOINC? A central manager which monitors and controls the client behaviour and configuration, if you read the suggestions threads it's also the aspect which will make f@h more donor friendly in the eyes of many people.

The stance has long been that the classic client with it's trayicon and gui offers just that, but only for one client. And this was sufficient as it was the only mature client. SMP and GPU where new kids on the block, their usefullness would not have increased with a central command center as much as it has improved ( lots! ) with improving the client/fahcore code it contains.

But I think we might be at the point where the upcomming smp3 and gpu3 have a mature base, and these features I requested could be the one of the next 'best possible improvements' as it in my opinion will increase the core of the project, the amount of donors who stick with it.

As security of data integrity is not involved, a central interface could be left to the 3rd party developers, but only if they get a proper interface to communicate with the clients.

[MtM]

Joseph we all know you're working hard on the server code and protomol core but could you please comment abit on my post above? If there are things you can not answer yet, just saying that would be enough for me, and better then a silent answer