Python API: Extension Module or Pure Module

Question

We have a C++ application, with C++ interfaces changing regularly during development. We also have a C API which is designed to be stable in spite of such changes; the C API is tighly focused on operations which client code might want to do.

Now, we want to add a Python API to make it easy for outside clients (or our own team) to create scripts controlling the application. Given the volatility of the C++ code and the by-design stable nature of the C API, it makes sense to build the Python API over the C one. I know the way to do this is to provide an extension module (implemented in C++) which will be usable in Python and wrap the C API functions.

What I am considering is whether I should make this extension module the Python API (and thus make it as Pythonic as possible), or whether I should keep this an "implementaion detail" of the Python API and build a pure Python module on top of it, which would serve as the actual external API. The extension module would then be a very thing wrapper around the C functions, resigning on being nice Python code.

I see the following pros and cons:

Python API is a C++ extension

This would be more efficient, as there are less intermediary layers. At the same time, it requires boilerplate code to provide Pythonic behaviour in C++, and the code is generally harder to write and maintain (reference counting, pointer tables etc.).

Python API is a module wrapping the C++ extension

This allows us to write Pythonic code in Python, which is easier and less error-prone. It would probably lead to a richer, more complete API as well, since adding functionality is easier and faster in Python than in C++. As a downside, there's runtime and memory overhead of the extra layer, as well as the need to maintain two separate bits of code for one thing.

---

My primary question is: are there any obvious upsides or downsides to some of the approaches that I failed to consider, especially ones that make one choice clearly superior to the other? Also, is there a widely accepted "best practice" for this type of thing? Or a "checklist" to help choose the correct approach for a particular situation?

Answer 1

You pretty much answered the question yourself, and in a way that confirms my own experience: A python wrapper is better in all accounts except for speed and memory efficiency. You are not telling us what sort of application is that so you'll need to carefully weigh how important those factors are for you. In my experience, most of the time they matter very little.

As for having to maintain two bits of code for one thing, you get that with both approaches anyway, and in the case of a C++ written API, it's much harder to maintain.