Providing same API implemented in multiple languages

Question

Our budding (currently internal) API deals with building information models, originates in our main application written in c++.

There is a small implementation of it in C# for easy interop with other CAD software exposing .Net APIs (mainly just a class library with de/serialization). This also covers python scripting through python.net.

We reached a point where it would also be useful to have a small Typescript implementation as well for use in web-gl based in-browser viewer app. (We tried unity with our C# API, but the non-packing GC that comes with IL2CPP turned out to be a hard limit in our use case, runtime generating and regenerating huge meshes based on user input).

Now, for just the "class library" part code generation is doable, where we parse the c++ classes and autogenerate POCs in the other languages. This makes sure at least our representation of the data is in sync across all theatres.

Problem is, if we wish to lift some logic into the API libraries like validation, the parse and generate method complexity increases exponentially, especially through diverse languages like c++, c# and js.

I currently see 3 paths forward:

1. Write the logic in only 1 language and hack it into the other environments (c++/il wrapper for C# and Emscripten for JS land). This works but introduces too many bottlenecks.
2. Write the logic in some common easy to parse and sufficiently vague DSL from which it is easy to generate all 3 languages. I haven't found a good already existing solution to this, rolling our own seems prohibitively time consuming.
3. Write the logic separately in all 3 languages utilizing each language to the fullest, ensure parity through rigorous testing with specification written in a common DSL (With something like Cucumber and Gherkin?) This is also time consuming, but we already spend time on testing which would be reallocated here.

What are the experiences with these strategies? Are there other major patterns for this?

Currently number 3 seems like the most promising, maybe because I have the least experience in it and of its pain points. Are there other / better frameworks for this than Cucumber/Gherkin?

EDIT: to elaborate on the architecture:

The main application is a big old windows desktop app (tightly coupled to mfc and all). In all API scenarios, communication is done through a binary file format. Eg.: in case of a .Net API call, the third party asks for the current model, which is serialized by the c++ code of our application, sent over through IPC to the .Net client, deserialized, modified by the client code, and serialized then sent back to the main application for evaluation.

All 3 API libraries would mainly deal with:

1. Serialization / deserialization of this binary format (this in itself is the easy part)
2. Provide class definitions to allow easy manipulation of the data in the target language (this is still not too bad with auto generated classes)
3. Hopefully provide some basic "in situ" logic for the data in the target environment so the API libraries do not have to constantly communicate with the main software, this is what prompted this question. One such common logic would be validation, another might be preview mesh generation.

Re.: How does a browser based view fit? We have a prototype solution where our users can upload their models, and with a simple Unity WebGL viewer app they can be inspected in the browser. Our C# API library is used to understand the uploaded file, and generate preview objects based on it's contents running inside the browser. The user can also make some modifications in the webgl viewer, and ask for re-evaluation of the model where we spin up a VM in the cloud with our desktop app, load the modified model, calculate it, and send back the new recalculated model into the webgl viewer. This works, but is resource and time expensive, so the goal is to replicate the validation logic in our C# library itself so we can pre-emptively validate the model inside the user's webgl viewer for example.

Answer 1

As you discovered, you'll have to have some degree of code duplication with respect to the class definitions. Once you start getting logic involved, though, I recommend that you write that logic in the C++ part of your codebase and compile it into a native DLL (not a managed DLL).

Native DLLs have the advantage of being accessible from almost every language out there. You can access it from C# using pinvoke. You can write a Python wrapper for it with ctypes or cffi. You can write a Ruby wrapper for it with ffi. Practically every language has a way to load a native DLL and call functions from it. That will reduce the amount of language-specific code to just what's needed to connect the library with your class definitions. It also means you can take your performance-sensitive code and optimize it once instead of having to re-optimize it for the quirks of each language.

Code running in the browser is a bit trickier since it can't load DLLs from the filesystem (not to my knowledge, at least). You can attempt to auto-generate it from the C++ code. The other option is to use the DLL on the server end like a web service (the client-side javascript passes data to the server, the server uses the DLL to process it, then returns the processed data to the client).

Answer 2

Can't comment yet because I don't have 50 Rep yet, so I have also included some questions in my response.

Restating main task :

Problem is, if we wish to lift some logic into the API libraries like validation, the parse and generate method complexity increases exponentially, especially through diverse languages like c++, c# and js.

First solution :

Write the logic in only 1 language and hack it into the other environments (c++/il wrapper for C# and Emscripten for JS land). This works but introduces too many bottlenecks.

This will be the ideal solution if it can be made to work smoothly somehow. What are these bottlenecks you speak of? If these bottlenecks can somehow be tolerated, the team can benefit from a fast development cycle with this solution.

Assuming option one cannot be made to work without headaches, other aspects to consider while taking a decision :

• What is the complexity of the code that we wish to lift (you mention "in situ" logic). If the code is huge, complicated, quickly evolving, then it can be hard to go with option 3. In this scenario, I would shift toward option 2. Otherwise, option 3 sounds like a good deal.