6

I'm planning on writing a C++ program which is heavily influenced by plugin modules. Initially I had only thought of making the APi available as an abstract class that others extend. After talking to someone they mentioned that I should think about making it available in a language agnostic way.

How do I effectively expose an API in a manner that any language can use it?

The solution that I could come up with was having the C++ API like I originally intended, for C++ directly and CLI languages (C#, F, etc) and then a sort of document and executable combination. Where expected input is described in the document and then the executable is started (via console/terminal) I would give parameters given as start up commands. This would theoretically work but doesn't sound like a standard solution too me. How is this done in other desktop software?

  • 3
    Are RESTful architectures relevant? Stack Overflow RESTful – Nick Keighley Oct 4 '17 at 10:56
  • Protocol Buffers is typically named as a silver bullet for API, because it reduces the message communication down to two parts: (1) the protocol, (2) the exchange of byte arrays. – rwong Oct 4 '17 at 10:58
  • Well I thought about doing a REST thing, but the issue I have with that is that I would have many different things that implement the API that I would have to track and tell what to do and they would all be sitting on the computer running waiting for a call for the duration that the program is running. One implemented API could be doing multiple things at a time. I guess I could pass along an ID for the work that is being done and reference that when asking for status updates and giving specific instructions but that would become complicated and error prone extremely quickly. – user3797758 Oct 4 '17 at 11:09
  • @user3797758: What kind of library do you have in mind? Please edit your question to tell more. What domain? What kind of users and usage? – Basile Starynkevitch Oct 4 '17 at 12:39
  • 1
    I take my "question seems obvious" comment back. After reading someone else's comment, I went back and reread the OP's question more closely and now I'm confused because they mention 'plugin' but then seem to describe something totally different for what they want. – Dunk Oct 4 '17 at 22:00
13

You cannot do that because some languages implementations could have an ABI different and incompatible with the C one.

On current systems, C is very common and most (but not all) language implementations have an ABI and calling conventions compatible with the one from C. Notice that it is a property of the implementation (your C compiler and your operating system), not of the programming language.

Also, some languages have different and incompatible views on control flow (look into call/cc and tail-calls in Scheme, Goroutines in Go, or backtracking and cut operator in Prolog, CLIPS rules, concurrent actor languages, ....). I'm not sure you'll be able to design something which easily fits all of them (and that could impact the FFI requirements of implementation of these languages).

If using C++, beware of name mangling and of exception handling. (e.g. C longjmp is not friendly with C++ exceptions). Dynamic loading facilities like dlopen and dlsym are relevant to name mangling. So prefer an API using extern "C" functions. libgccjit could be inspirational (it is coded in C++ but has a C friendly API) and perhaps useful (you could consider runtime generation of glue code).

Memory management (notably with garbage collection) is also an issue. Study for examples foreign function interface of Ocaml and of SBCL and of Lua and of Guile. Look into libffi. Some languages also want serialization or persistence facilities (even for foreign data). Read also about dynamic software updating.

You could provide some reflection facilities (e.g an API to query your API, e.g. like GTK introspection). You might try to provide a generic closure mechanism like in GObject-s.

You could use (or customize or adapt) code generators like SWIG. You might consider compiler plugins (e.g. GCC MELT extensions).

If you have a wide and complex API (hundreds of public functions or data types) consider at least providing a machine readable form of it, e.g. some database or XML or JSON from which automatic glue code could be generated.

There is no silver bullet.

NB. Better make your stuff free software. You might get outside contributions and you'll need outside help to interface your thing with many programming languages, including some that you don't even know. Open source is a good way to counter leaky abstractions (since you and other contributors can dive into the implementation source code). Given the variety of programming languages, you will need outside help (as soon as your API is successful)


why the question don't make any sense

I asked several times the OP to give much more context and motivation, and to explain the domain for which the library is written, and the context in which that library would be developped. Sadly I've got no answers at all.

The applicative domain (e.g. high speed frequency trading, software for dentists, selling machine, weather forecast, image processing, speech recognition, word processing, social network, static source code analysis ...) matters practically a lot, because it defines what programming languages are likely to be considered and what programming paradigm is probably used. For instance, an image processing library is very unlikely to need to be interfaced to Agda or Coq, but probably would be used from C or C++ applications. Business software is often written in Java (and older ones was in Cobol) so needs to be interfacable to JNI or JVM. Static source code analysis software is very likely to use some theorem prover (so Ocaml or Common Lisp or Haskell is important to them, and they need to be garbage-collection friendly). And so on. Sometimes, inter-process communication (including RPC, MPI, web services, RESTful applications) can be used, but at other times it is not efficient or not convenient enough. Details and context are very important and we don't know them.

The question mentions "desktop software", but that just means today something having a native (non-Web) GUI used with a mouse and a large enough color screen (the dentist software, the software for managing my bank accounts, your favorite game software, the word processor, a web browser are all desktop software, but their main commonality is just having a GUI). Using Qt (a very powerful GUI cross-platform toolkit for C++) facilitates the development of such things. And quite often a library used on desktop (think of XML or JSON libraries like Xerces and JsonCPP, machine learning libraries like TensorFlow, HTTP client libraries like libcurl, numerical computation libraries like BLAS or GMPlib, etc...) don't even care about GUI (but it could care about response time -having functions returning in less than 0.4 seconds- and thread friendliness), because the application (not a reusable library) would care about GUI itself.

The question mentions plugins, but these are simply compiled software components which get dynamically loaded at runtime (thus increasing the virtual address space of the process loading them), using e.g. dlopen on POSIX (and probably LoadLibrary on Windows, which I don't know at all). Notice that the JVM don't use (Java-coded) plugins but dynamically loaded classes, and speaking of plugins don't make sense with JavaScript or with Common Lisp (or most languages having a compile or eval primitive). So the mention of plugins don't means much (both Firefox browser and GCC compiler accept plugins, for very different reasons).

Without a lot more additional details (including applicative domain, context, motivation ...) the current question don't make any sense. I offer to delete my answer if that helps in closing the question in its current form.

PS. I don't know and never used Windows (but I use Unix since 1987, and Linux since 1993), but that is an unimportant detail.

  • 2
    I would like to add that most languages, even if their ABI is not compatible with C, will still be able to use it with special syntax(to annotate the types and functions) or with some FFI library. This is because C libraries are so common - and also very simple. No need to support classes, overloading, exceptions or managed memory - just locate the function by unmangled name and pass it some arguments. – Idan Arye Oct 4 '17 at 9:29
  • 5
    Reading your answer I went like "yes, mmmh, yeah, probably, correct" and then ... I saw that teddy bear logo and for god's sake, yet another rant on free software. Maybe it's just me, but you could use those three lines to actually answer the question instead of (sigh) proposing to patch leaky abstractions with free software. – user44761 Oct 4 '17 at 10:24
  • There is no generic answer to such an ill-defined question. And there are many programming languages, some with implementations having weird calling or FFI conventions; I don't even believe that a universal API or library is possible – Basile Starynkevitch Oct 4 '17 at 10:42
  • 2
    I have to agree with @Tibo: This strikes me as a little bit of sort-of topical information as an excuse to (yet again) post your favorite rant, regardless of its being irrelevant to the question. – Jerry Coffin Oct 4 '17 at 14:51
  • 3
    @BasileStarynkevitch There is a huge difference between writing something for every language implementation and building something language agnostic. Also, what the hell has Open Source/Free Software to do with this? You can't solve abstraction issues by just pushing it to Open Software. This makes no sense in my book. The bulk of the answer is good but those ending lines are like a kick on the bucket of milk. If the question makes little sense, refrain to answer and ask for clarification. – T. Sar - Reinstate Monica Oct 4 '17 at 19:55
16

The accepted answer is more or less correct, but doesn't offer you anywhere to go if language independence is actually a burning issue for you. C++ is fairly callable from other languages (C#, Python, etc), and from the tone of your question it sounds more like a "nice to have", but let's pretend otherwise for a second.

If platform/language independent communication is important to you, you should be using a service-oriented architecture. You should provide your C++ code as its own self contained service, which can respond to requests over technology agnostic network protocols.

This is pretty common in the world of server applications and has a lot of benefits to do with architecture, scalability and protecting your applications from nasty memory bugs (+ stack overflows, etc) in other people's code.

It's a little bit more unusual for desktop apps though. I have seen it with the Rider IDE, where they mainly use it for the architectural benefits of keeping their ReSharper product separate from their IDE product.

This is ultimately how you make things agnostic, but also represents a serious architectural change that has a lot of advantages and disadvantageous in multiple areas. Look into it and decide.

8

Although none is literally 100% universal, there are a number of ways to make the API easily usable from a fairly wide variety of languages (more than most people usually care about, anyway).

The most obvious (and ubiquitous) form would be RPC, such as gRPC or Thrift (or, if you're using Windows, COM).

These all work reasonably similarly: you define the API itself in an interface definition language (IDL). You then use a compiler to generate code for the client and/or server side of this interface. This will typically give you the skeleton of a server. You fill the actual functionality, compile it all up, and off you go.

  • Also, XPCOM. Much like COM, but open. – Basilevs Oct 10 '17 at 18:40
5

While I agree with the other answers in principle, I think the OP is expecting at least some starting points which can help OP's further search for an answer.

In-process

  • Important consideration

    • Native code is architecture-specific. This means 32-bit and 64-bit can't mix in the same Windows process.
    • While pure .NET IL can be executed in either 32-bit or 64-bit environments, native code must be compiled individually for each architecture.
    • If these limitations are show-stoppers, you must use out-of-process approach instead.
  • Win32 DLL

    • Export C-callable functions.
    • Can pass integer values, floating point values, arrays of primitives (including byte arrays), etc.
    • Be careful with "structs", make sure two pieces of communicating software agree on struct field alignment for each struct.
    • .NET users can use P/Invoke to call DLLs. This can be converted into a .NET wrapper, which can completely hide the native aspect of the underlying implementation.
  • Microsoft COM
    • Mature, recommended
  • .NET Interop using C++/CLI
    • Recommended if most users are on .NET

Out-of-process, same machine, non-communicating (no information exchange in the middle of execution)

  • Command line application.
    • Communication happens via command-line arguments and files.
    • Using I/O redirection on Windows is not recommended because it is unreliable except for very simple ones. The documentation mentions deadlocks as a possibility.

Out-of-process, same machine, communicating

  • Choice of running as Windows Application or as Windows Service
  • Command line application plus any choice of inter-process communication. Refer to inter-process communication on Windows documentation.
    • Shared memory or inter-process memory-mapped file
    • Inter-process synchronization primitives (only for synchronization or semaphore needs)
    • Inter-process pipes
    • Over the file system. Synchronization (hand-off) must happen via one of the other means. Alternatively, repeatedly scan a folder for file changes at periodic intervals.
  • WCF

Out-of-process, different machines, communicating

  • Architectures
    • Server client architecture
    • Distributed architecture
  • Writing an web service that runs under IIS
  • Running as a Windows Service that communicates over TCP/IP
  • Running as a Windows Service that communicates over WCF
  • Your answer is exclusively Windows centered, and Windows was not exactly mentioned in the question (even if F# or CLR is Windows mostly) – Basile Starynkevitch Oct 4 '17 at 10:47
  • 5
    @BasileStarynkevitch Since you have already listed options for Linux, I think my answer complements the Windows part. Though, I sincerely believe OP is indeed having Windows in mind. Also, my answer contains some options that are OS-agnostic as well. P/Invoke works for non-Microsoft CLI implementations (e.g. Mono, .NET Core, etc.) Let's not hijack OP's question for personal software-political expressions. – rwong Oct 4 '17 at 10:56
  • Yes, but you could at least mention Windows in your answer. Few things apply to Linux unchanged. – Basile Starynkevitch Oct 4 '17 at 10:58
  • 4
    @BasileStarynkevitch did you not see the word "Windows" prefixed to every phrase where it is applicable? – rwong Oct 4 '17 at 11:00
  • 1
    @user3797758 First spend some time with higher-up stakeholders to determine whether future requirements (e.g. next 2 years) will make out-of-process (or, over-the-network) necessary. If in-process is okay, then I'd recommend P/Invoke to a DLL (Windows) or SO (Linux). Basile's answer mentions a lot of tools that can generate wrapper / shim code to make this easier. – rwong Oct 4 '17 at 11:11
4

To bring an "old-fashioned", but quite mature technology into the discussion:

CORBA allows you to define object-oriented APIs in a language-independent way (defines its own IDL = interface definition language). Language mappings are available for really all major languages (and even many exotic ones).

It supports local and remote inter-process communication (that's the typical use case), but server implementations can also exist in-process (then using direct method calls). The inter-process communication protocol is binary-based and quite fast.

At our company, we're still using it for performance-critical cases, and interoperability always worked (using mixtures of Java, C, C++, C# and CommonLisp). We've used CORBA for interfaces to partner companies, often not even knowing what programming language they used.

And maybe you've already used it without knowing. E.g. Java's RMI/IIOP is effectively CORBA.

  • yep i did though of corba as soon I red the title, whatever we think of CORBA, it definitively match the requirement of OP. – Walfrat Oct 5 '17 at 7:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.