I am working on a C library (SlipRock) for interprocess communication.

The library currently exposes a simple, blocking API. This is easy to use, makes misuse (relatively) difficult (this is C after all), and will be (relatively) easy to implement cross-platform, once I get around to implementing the Windows port (which shares essentially no common code).

However, I am worried that the blocking API might cause problems in certain use cases. Nevertheless, I am very worried about providing a non-blocking API that is easy to use and doesn’t require the use of heavyweight libraries like libuv or libevent.

Bindings to languages that naturally do pseudo-blocking (blocks the green thread) IO (such as Go, Erlang, or GHC Haskell) can always just implement the higher-level API (or even the entire protocol – it isn’t that complex) in the higher level language. Similarly, it is possible to implement the parts of the protocol that are not by-nature blocking (listening on a named pipe/AF_UNIX socket and connecting to an AF_UNIX socket, as well as the following exchange of passwords) in the higher-level language.

Should I provide a non-blocking C API, or should I just provide the 2 or 3 functions thatbindings would need to use as the basis for the API?

The way I was thinking of implementing the non-blocking API was by having a state machine that the user is required to manually invoke. But that requires the user to have access to the underlying IO handle – which is the intended return value! So I either need to break encapsulation or tie myself to an event loop implementation if I want to expose an async API for this not-very-performance-critical code.

  • If you provide a non-blocking API, how will you implement it? Will the user register a callback function? Will you support Futures or Promises? Will you allow continuations? If you don't provide a non-blocking API, can you rely on the user properly implementing asynchrony himself? Would you provide guidance for such? Do C calling conventions (or the C language itself) provide adequate support for asynchrony? Would you recommend that your users employ some external library to implement the asynchrony over your synchronous methods? Commented Apr 19, 2017 at 22:39
  • @RobertHarvey C most certainly does not provide adequate support. At least not without tying myself to an event loop implementation, which I really don’t want.
    – Demi
    Commented Apr 19, 2017 at 22:48
  • 2
    If you're dependent on file descriptors, what's wrong with using select(2) or poll(2) (both portable to any POSIX.1 system, BTW) with a zero or infinite timeout depending on whether or not the user wants to block? If you're providing a non-blocking interface, then by definition you're leaving responsibility for how it's (mis)used to the callers.
    – Blrfl
    Commented Apr 20, 2017 at 11:48
  • 2
    Microsoft pretty clearly didn't get the memo about that.
    – Blrfl
    Commented Apr 20, 2017 at 12:43
  • 1
    The C-compatible Win32 API supports asynchronous I/O, see e.g. here and here.
    – ChrisW
    Commented Apr 20, 2017 at 19:34

3 Answers 3


You might also provide low-level operating system specific hooks to enable (or facilitate) non-blocking IO

For example, for Linux and POSIX system, you could provide a way to get the active file descriptor sets (and leave your user to call poll(2) on them, probably providing the user some hooks to do the actual I/O when poll detects available IO), and perhaps to be notified when that set is changing. The multi interface of libcurl could be inspirational (and perhaps also 0mq). And you might even try to target aio(7)

Perhaps you might design your library above some existing event loop library (like libevent or libev).

Don't forget to document wisely the asynchronous aspects of your library, since they are always tricky. You could also document how to use it in a multithreaded application.


Whether you should provide a non-blocking API depends on the needs of your users. I agree that forcing users to add a dependency to a heavyweight library is a bad idea. You might put the non-blocking API in a separate library.

On the other hand, I don't agree that you must break encapsulation. I don't know the details of your API, but I am sure that you can hide the file handle in some way.

Did you ask actual users of your library to see what they need? If not, why add a non-blocking API?

  • 1
    My library has no users – yet. It is very incomplete and nowhere near production-ready. That will need to wait at least until it works on Windows.
    – Demi
    Commented Apr 25, 2017 at 17:31
  • @Demi Perhaps you could speak to potential users? There must be a reason you are creating this. Commented Apr 26, 2017 at 18:33
  • 2
    Because so many systems get it wrong. Zinc (Scala compiler). LLDB. SLIME. ZeroBrane Studio. Py4J. All of the trivially vulnerable to local (and possibly remote!) code execution exploits. All of them. That needs to change.
    – Demi
    Commented May 2, 2017 at 3:37

If you do want to provide a non-blocking interface, there are only a few ways of doing it:

  1. Embed the client code in your event loop.

    You provide your own event loop/dispatcher. Add an interface for the client to add their own sockets, timeouts and event callbacks.

    You run your event loop, manage your internal async state, call the client callbacks when their sliprock write completes, or when a select/poll/whatever event happens on a socket they registered with you

  2. Expose your fds to the client event loop.

    Require the client to call your handler when something happens to one of your internal fds.

The hybrid option is run your own event loop in a dedicated thread which handles your sockets and your async state, and communicate with the client code via a single exposed socketpair or eventfd which they can poll in their existing event loop.

In both the first option and the hybrid one, you avoid the requirement for your client to manually invoke your state machine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.