Design pattern to create a shared lib architecture

Question

I am making a renderer as a hobby, one thing I thought to try is making the low levelAPI be dynamically swappable, i.e. you could have an opengl or vulkan backend and switch between the two without needing to recompile or even relaunch the application.

To achieve this I am making 2 binaries, one is the executable, the other is the rendering code.

Let's call them main and rendering. For this to work, there is a core header Core.hpp that defines all the C like functions and message PODs that are used to pass information between main and rendering.

I have one issue, I need to create objects/class instances in rendering. Due to a series of constraints these are C++ instead of C, which means they cannot easily pass the barrier between the 2 binaries.

Because these objects need to live for as long as the rendering code is available, my current approach to handling their memory is to allocate on the heap into global pointers that are deleted when the main calls a deinit method. For example, the memory allocation object would be initialized like this

MemoryAllocHandler * mem_handler;

void Init()
{
   mem_handler = new MemoryAllocHandler();
}

void deinit()
{
    delete mem_handler;
}

This way I can expose a C like API that receives parameters and uses these objects to handle rendering (fro example allocating OpenGL buffers).

This works, but I hate it. If I was statically linking rather than dynamic linking, I would not allocate on the heap, I would allocate on the stack and return these objects, hold them in some structure initialized at the highest possible scope and rely on RAII idioms to have its memory freed.

For example:

MemoryAllocHandler Init()
{
   return MemoryAllocHandler();
}

Is there a way I could do something like the above while still dynamically linking? Take into account the prior is A C++ function with a C++ struct return type so name mangling is not just present but necessary.

Answer 1

Unfortunately, it's impossible to allocate things on the stack without knowing their size at compile-time - which is what you want. At least, it isn't possible in C or C++ in a convenient way. You can use alloca but it's really ugly.

But you can still get RAII with heap-allocated objects. Just use std::unique_ptr:

// in DLL
std::unique_ptr<BaseMemoryAllocHandler> Init()
{
   return std::make_unique<VulkanMemoryAllocHandler>();
   // VulkanMemoryAllocHandler derives from BaseMemoryAllocHandler
   // Make sure BaseMemoryAllocHandler has a virtual destructor
}

// in executable
int main()
{
    std::unique_ptr<BaseMemoryAllocHandler> mem_handler = Init();
    std::cout << "hello world\n";
    // mem_handler is automatically deleted at this point
}

Answer 2

Seen as this is a C++ library, why not use a third common library that encapsulates the abstractions which the two other libraries implement, and also contains the long lived common objects?

This side steps the issues of name mangling (assuming common versioning), and makes it so that long lived objects are handle by the one library that is not swapped out.

A naive C wrapper around this is relatively trivial.

Answer 3

How would I do that:

Rendering DLL (rendering) can be written in C++ but must have a C API like that:

// rendeing.h

VERTEX_BUFFER* CreateVertexBuffer(size_t size);
void           DeleteVertexBuffer(VERTEX_BUFFER*);

On the client side (main) if you wish you can wrap it into some C++ API like that (RAII):

class VertexBuffer
{
public:
    explicit VertexBuffer(size_t size)
    {
        buf = CreateVertexBuffer(size);
    }

    ~VertexBuffer()
    {
        DeleteVertexBuffer(buf);
    }

    // here go C++ complexities to disable copying and stuff like that.
}

Answer 4

Hmm. You could force the RAII structure by making a "shadow main" function that takes a callback. Even a C-style "pointer to function taking no arguments and returning void" callback.

Then your shadow main function can create the objects and call the callback. The library consumer runs their entire program in the callback. On normal exit, they return and unwind through your RAII code.

The downside of this is that it forces a particular structure on programs that consume this library. It makes it very difficult to use inside another library as well, unless that library also uses this pattern.