3

I have a program that loads a bunch of "addons" (different meters that measure and display system values).

The "addons" have "grown" over time and the code of what addons to "activate", and how many of each (if there is more than one of the "device" in the system (cpu's, net-interfaces, etc) is hard-coded into the master. Ug (and ick).

What I want is for each addon-meter to place it's name and init-routine-addr into an array as they are loaded. Right now, what meters that are loaded are fixed in the master at compile time. What I want is for the possibility of dynamic loading, but more importantly, for external "script-extensions" (that could measure "almost anything" and rely on the master to display the collected data (like a script that would read outdoor temperature via some 3rd party device). In other words -- long after the master has been compiled, I want to provide the ability for other addon meters to be added as long as they supply a specific callable API.

Thus, as a new extension is added, I need some init routine to be called as it loads that will place the new meter-name and routine to be called in a class-singleton that has an array of the extension names and their init-routine addresses (as in a dynamically loaded extension).

Then when execution starts, each extension will already have have "registered" itself with a central list that the "master" can look at and know what to call.

In Perl, I've used something in a "BEGIN{...} block" in loaded extensions to call a Plug_Common->register("plugin name", \%Plugin_Description_Data); so when the main routine started it would look in the plugin-registry to see what was available, dynamically, at runtime.

The question is, how do I do this in C++?

I've setup register routine, but I've no idea how to have the extensions get their "name & address" registered before the main-master runs. Theoretically, since some of the extensions could be added via scripts long after the main-master was written -- it wouldn't be able to know their names in advance.

Off the top of my head, I can only think of a "user-edited" static text file, that might be able to have the master, "somehow(?!)" call each extensions init and 'get_config' routine. Then I can get rid of all the 'extension/meter-specific'-knowledge out of the the master. I don't like the text-file idea as it's too prone to user-error -- having something being able to automatically call a register-function as it is loaded, seems much less likely to have problems like the static-config file being out-of-sync with what extensions can be used on the system.

Any ideas? (I'd rather not have to build in a copy of something like Perl to handle the dynamic "loadability" of modules...to me, that would be 'ugly'...

Seem a bit unreliable for the master program to try to look through some symbol table at runtime -- especially with C++'s strong typing. I keep coming up with flaws in virtually every method I think of. :-(. Anyone know a better way? ;-)...


Update:
Had a thought (well more than one, but this one is a bit unconventional, but might work) What if each of the plugins declared a shared pointer to a global list in the master?) If the shared pointers were used in module level inits, then wouldn't the shared 'use count' be incremented as the modules were loaded?...hmmm... I don't know yet how to get the modules' addresses into the main array (without them getting any execution time), but if I can figure out a way for an initializer statement using the shared_ptr, then since loading it a non-multithreaded (at least at this point) if that pointer could be leveraged to increment and store the address of an init function, in a global array, that might give me my dynamic config? Right now, the only thing I know for sure is that shared pointers do have an accessible 'use_count' -- which would only help me in determining how many plugins had registered. Maybe there is no way to make this work, but just a thought (at 5:45am after I've been up all night... so, may be completely flawed and I just don't see it yet...;-))... Anyway, thought I'd add my latest thoughts; For scripts I'm thinking of splitting off that type of plugin into a separate handler that would have to read some config... I don't really like it, but it does isolate those things that wouldn't load @ program start time... Anyone think it is possible to leverage the shared-pointer to do this?

2

3 Answers 3

5

In most plugin systems in C and C++, the loading of plugins works like this:

The main application specifies a path where the plugins should be located (or allows such a path to be configured). On startup, the application looks for either dynamically loadable libraries (DLLs on Windows, .so on Linux, in case of compiled plugins) or scripts (in case of interpreted/scripted plugins) in the plugin path.
For each plugin that it finds, the application tries to call an init/registration function that allows the plugin to register itself with the application. For interpreted/scripted plugins, this could be the main body of the script.

To support interpreted/scripted plugins, the application needs to incorporate a scripting engine that understands the language that the plugin is written in.

When writing a compiled plugin, one of the rules is that you build it as a dynamically loadable library and that you provide the initialization function with the exact name and signature that the main application expects.
Furthermore, any classes that the plugin provides must be derived from an interface specified by the main application for plugins that provide a certain service.

For a scripted plugin, the requirement is that the initialization function performs the initializations and registrations that are expected by the main application.

8
  • "When writing a plugin, one of the rules is that you build it as a dynamically loadable library and that you provide the initialization function with the exact name and signature that the main application expects." I'd even extend that a bit and suggest that you keep the interface as generic as possible, if third party devs are supposed to write plugins. Do not exchange or pass any complex types (like STL classes or classes in general), since things might get messy and simply crash if people try to mix libraries built by different compilers and/or versions.
    – Mario
    Mar 12, 2015 at 7:51
  • 1
    @Mario: That it is built by a compiler that uses the same ABI is not specific for plugins, but holds for any library. Mar 12, 2015 at 11:08
  • 1
    Sure, but stuff like the layout of STL types isn't specified as part of the ABI, which is problematic when STL code expecting different layouts is inlined in different modules.
    – Useless
    Mar 12, 2015 at 11:28
  • Of course this is true for pretty much any dependency, but plugins are a very common use case where it's hard to ensure consistent versions, no matter whether it's basic runtime, STL or something different. Add hobbyist devs to the mix and it's probably a matter of hours for issues to arise.
    – Mario
    Mar 12, 2015 at 12:01
  • Um, a plugin, in my environment involves the user being able to write it in 'shell', or perl, -- as long as it passes the labels on init, then can provide a data stream periodically, that's about it. I don't want people to have to know how to write and compile a DLL on windows or a .so on *nix. While I might provide a binary API, I was thinking of something that can talk to to stream-based plugins. Example: I have a perl program now that displays temp ~ every 5 seconds that it gets from calling an IPMI func. I could see a modified version supplying that to the 'master'.
    – Astara
    Mar 12, 2015 at 12:11
3

The question is, how do I do this in C++?

You will not be able to map the Perl initialization in C++ without making it explicit.

Components for a plugin system in C++:

  • define your plugin interface as a header file (will probably contain the declaration of a plugin initialization function/class, a configuration base class interface and class factory, a plugin base class interface and a plugin class factory (or similar)). This header file will be used by both plugins and the master application, to understand each other.

  • create an empty plugin that implements this interface fully and keep it for reference [good for maintenance, development of plugins, proof of concept and documentation example].

  • write high-level client library for the master application, for working with plugins. Test that it loads your empty plugin [good for making work with plugins easy in the master application].

  • describe plugins in a simple text/json/xml file. Keep the format as simple as possible. This will allow you to add different plugins into your application, at runtime (because it is much easier to edit a line in a configuration file and add a comment, than to remember where you have to move dynamic libraries on your hard drive.

Document steps for developing a new plugin based on the empty one, and keep the steps up to date in the docs.

Edit (addressing comments):

My prototype code already has the interface being pure-virtual, so I can use dynamic_casting at run time to get at the needed data

Don't do that (the dynamic_cast thing). Instead, define a richer protocol between the application and the plugins. Define (for example) an event-based protocol in the virtual interface, and call through that:

struct your_plugin_class
{
virtual void on_init() = 0; // called from the application, on initialization
};

My main problem right now, is how to have plugins "register" themselves (in so much as to putting their init-function in a call-array, that the master can use to dynamically adapt to whatever list of plugins have been inserted on a given run -- without recompilation.

Create a plugin manager that opens files at a given location, with a pattern-matching name (for example, "do-this-and-that.your-app.plugin.so" - something that fits nicely into a regular expression). Within this library, simply call a custom class factory.

You could even have the plugin manager be an active object and keep an in-memory list of the loaded plugins. Periodically, you iterate the plugin files, and if there are differences, load them on the active object thread. Your application would pick plugins at run-time.

5
  • not sure what you mean by not mapping the perl init in C++ w/o making it explicit? Why can't I call a perl script that returns a measurement every second (well the IPMI overhead might be one reason, but aside from that)... My prototype code already has the interface being pure-virtual, so I can use dynamic_casting at run time to get at the needed data. I have 2 classes of plugins that I plan to handle -- one "built-in" plugins (current meter set -- that I might convert to "dlopen" functions -- but that is straining portability. The other is to provide for calling an arbitrary script.
    – Astara
    Mar 12, 2015 at 12:16
  • by arbitrary script, I mean(also) arbitrary executable... something that can give me the fields (or that might be part of a data-blob read at init time), and some means of calling it periodically to get the data. My expectation was that I might have to all the 2nd type of plugin in a separate process (perhaps / plugin) so a slow plugin wouldn't "plug-up" my whole data collection routine... did I mention I also would like this to be OS-agnostic?... ;-)
    – Astara
    Mar 12, 2015 at 12:19
  • My main problem right now, is how to have plugins "register" themselves (in so much as to putting their init-function in a call-array, that the master can use to dynamically adapt to whatever list of plugins have been inserted on a given run -- without recompilation. I want to stay away from a need for a config file that is separate from the actual list of plugins that are configured to run -- as they can get out of 'sync' -- I want the list of plugins to be determined dynamically based on registration (but that requires they be called by something @ load time) -- that's my problem.
    – Astara
    Mar 12, 2015 at 12:24
  • Right now, I'm working on using a global vector that includes the 'classname' and a handle to an instance of the class. That can be initialized at loadtime by having a class created at the end of the ".cc" file, so the master won't have to look anywhere for the binaries meters that are already in place, and will know which to call based on which ones added an entry to that array. --- Scripting will take a special module that can work with STDIO and some sort of config files. those might be places to use xml.
    – Astara
    Mar 12, 2015 at 20:48
  • Basically, I didn't want the 'master' to have to even know what classes (network, disk-io, cpu...etc) let alone, as it does now for some of the meters, doing device probing for some of the meters so they can be called with custom params. IMO, that needs to go so the master can deal in generics. The main sticking point was getting the list of configured classes and their config-data somewhere where the master could see it at the beginning of execution without needing to read config files (though it does read the Xdefaults file for meters it knows about), but that's too little separation.
    – Astara
    Mar 12, 2015 at 20:55
1

On Linux, if compiling code with GCC, the static data's constructor is run at dlopen(3) time. See also the visibility attribute and the GCC specific __attribute__((constructor)) for functions.

However, you should define a plugin convention: which names (for extern "C" C++ functions inside plugins) are used, with which signatures, in what order, etc.

For example, you could decide that every plugin should have a function extern "C" void my_plugin_init(void); which your program would run (after having dlsym-ed it) when loading a plugin, etc... That my_plugin_init might register some other functions (perhaps by adding them to some global std::map<std::string,std::function<BaseClass*(int)>> variable, etc....)

The Qt Plugins framework could inspire you.

1
  • I don't want to go with a custom interface, as I'd like the resulting code to have as few extra dependencies as possible. Right now, it depends on 'X' for graphing, but that's it. While I might think about dlopen, I'm pretty sure that wouldn't work across the platforms that it currently works on (BSD, Solaris, Irix(sgi), HPUX are a few that I've seen references to in the code). I have to also be careful not to use something GCC specific, as I believe other compilers are used on other platforms. The more I special requirements I add, the less portable it will be.
    – Astara
    Mar 12, 2015 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.