Providing a non-blocking IO API in a C library

Question

I am working on a C library (SlipRock) for interprocess communication.

The library currently exposes a simple, blocking API. This is easy to use, makes misuse (relatively) difficult (this is C after all), and will be (relatively) easy to implement cross-platform, once I get around to implementing the Windows port (which shares essentially no common code).

However, I am worried that the blocking API might cause problems in certain use cases. Nevertheless, I am very worried about providing a non-blocking API that is easy to use and doesn’t require the use of heavyweight libraries like libuv or libevent.

Bindings to languages that naturally do pseudo-blocking (blocks the green thread) IO (such as Go, Erlang, or GHC Haskell) can always just implement the higher-level API (or even the entire protocol – it isn’t that complex) in the higher level language. Similarly, it is possible to implement the parts of the protocol that are not by-nature blocking (listening on a named pipe/AF_UNIX socket and connecting to an AF_UNIX socket, as well as the following exchange of passwords) in the higher-level language.

Should I provide a non-blocking C API, or should I just provide the 2 or 3 functions thatbindings would need to use as the basis for the API?

The way I was thinking of implementing the non-blocking API was by having a state machine that the user is required to manually invoke. But that requires the user to have access to the underlying IO handle – which is the intended return value! So I either need to break encapsulation or tie myself to an event loop implementation if I want to expose an async API for this not-very-performance-critical code.

Answer 1

You might also provide low-level operating system specific hooks to enable (or facilitate) non-blocking IO

For example, for Linux and POSIX system, you could provide a way to get the active file descriptor sets (and leave your user to call poll(2) on them, probably providing the user some hooks to do the actual I/O when poll detects available IO), and perhaps to be notified when that set is changing. The multi interface of libcurl could be inspirational (and perhaps also 0mq). And you might even try to target aio(7)

Perhaps you might design your library above some existing event loop library (like libevent or libev).

Don't forget to document wisely the asynchronous aspects of your library, since they are always tricky. You could also document how to use it in a multithreaded application.

Answer 2

Whether you should provide a non-blocking API depends on the needs of your users. I agree that forcing users to add a dependency to a heavyweight library is a bad idea. You might put the non-blocking API in a separate library.

On the other hand, I don't agree that you must break encapsulation. I don't know the details of your API, but I am sure that you can hide the file handle in some way.

Did you ask actual users of your library to see what they need? If not, why add a non-blocking API?

Answer 3

If you do want to provide a non-blocking interface, there are only a few ways of doing it:

1. Embed the client code in your event loop.

   You provide your own event loop/dispatcher. Add an interface for the client to add their own sockets, timeouts and event callbacks.

   You run your event loop, manage your internal async state, call the client callbacks when their sliprock write completes, or when a select/poll/whatever event happens on a socket they registered with you

2. Expose your fds to the client event loop.

   Require the client to call your handler when something happens to one of your internal fds.

The hybrid option is run your own event loop in a dedicated thread which handles your sockets and your async state, and communicate with the client code via a single exposed socketpair or eventfd which they can poll in their existing event loop.

In both the first option and the hybrid one, you avoid the requirement for your client to manually invoke your state machine.