4

I am sure that this is a common design pattern, but I seem to have a blind-spot.

enter image description here

I have a requirement that the function call from Application to Service be blocking, but service needs to do something asynchronous.

Well, I can handle that, BUT, there can be multiple applications and there is only one service.

While the service should block the application, it should also be able to handle multiple parallel requests from different applications.

I suppose I need to spawn a new thread for each application’s request, but how then can I return from the thread to application at the end?

I don't mind an assembler insert to store the application’s return address & pop it later, but is that really necessary?

Do I need to do anything special in C++ to mark the service function as re-entrant?

Can someone please lift my blindfold?


[Update] There is no correlation between the applications. Also, only the service would be allowed to spawn a thread, not the applications. So, I am thinking of a service main spawning a bunch of service threads, but I don't see how I can can handle the function call blocking & return.

  • Many people will suggest to not do sync over async. – ratchet freak May 21 '15 at 12:02
  • Indeed they will. And I wholeheartedly concur with them. However, I have been given a requirement and I must either fulfil it or declare it impossible. Discussion is not possible in this case, but a statement that it is not possible might be. Preferably would be an indication of how to code this, or an alternative architecture which still meets the singe requirement of non-blockingness. – Mawg May 21 '15 at 12:05
  • is this in one process (service is a library and application has multiple threads that call into it)? – ratchet freak May 21 '15 at 12:17
  • Service is not a library, just a function which the Applications can call. – Mawg May 21 '15 at 13:01
  • 1
    @Mawg: You need to clarify the "do something asynchronously" part in your diagram. What kind of coordination requirements exist between requests made by different applications? If no coordination is needed, then the simplest design of one-thread-per-application (not per-request) is sufficient. – rwong May 21 '15 at 13:45
4

You have to use a blocking construct on a per-request basis.

This is called Futures (programming)

Each request will have its own Future. Once the request processing is started (possibly in a separate pool of threads, as you have described), the caller will be blocked on the Future's fulfillment, or failure.

When the result arrives, the Future is unblocked, and the call will return to the application.

When using this pattern, one must take great care prevent deadlocks and zombies. In other words, the Future needs to be unblocked at some time, whether the result is successful or failure (and in case of failure, throwing an exception back to the application is a fair game) - the call must not just hang there indefinitely.

As to thread-pool design:

  • There will be at least one thread per simultaneous requester (i.e. application), which will be blocked during the request.
    • If the applications and the service are inside the same process, these threads are the same as the application threads, which will become blocked.
  • There will be a separate thread pool, requiring as many threads as is necessary to do the work between the service and the internet. These threads are in addition to the request-accepting threads.
    • The exact number depends on how the work is done. It is possible to use asynchronous pattern (reactor) here, which might reduce the number of threads needed, but that will not have any effect on the number of request-accepting threads.

If the request from Application to Service occurs over network, there is a decoupling between Application threads and Service threads (in other words, a blocking of an Application request does not involve a "thread" at all; just a non-response from a network connection.) In that case the Service can also use reactor pattern, which means you can further reduce the number of threads.

  • I have awarded this the answer as it does indeed answer the question as posted. However, I just realized that this is a C++11 feature and my company is frozen at C++03, so I have posted a new question to find another way to do it. Anyone who is interested can look at stackoverflow.com/questions/30477812/… – Mawg May 27 '15 at 9:07
  • 1
    @Mawg, std::future is a C++11 feature, but this answer describes futures as a general pattern, not a concrete class that implements that pattern. – Jonathan Wakely May 27 '15 at 9:18
1

Since the application is blocked while it is in the service call, every application already must have its own thread if there are to be multiple service calls at the same time.

Assuming the service call simply starts the async operation, maybe does something else, and then blocks until the operation finishes, I don't see the problem. Since every call to service is on its own thread, you can simply block that thread.

1

The short answer to your question is that you don't return from the asynchronous code. Instead you pass a context object to your asynchronous code to send a response back when you finish processing the request. This is done mostly in two forms:

You are using some server framework that already takes care of the async for you:

void init() {
    registerHandler("requestType", [](ReqType req, Response resp) {
        // ..
        resp.send(data);
    });
}

Or if you handle client requests in a blocking event loop you can achieve asynchronous by using multi-thread, like this:

while (true) {
    // ...
    Req req = recv();
    thread t(processReq, req, ctx);
}

void processReq(Req req, Context &ctx) {
    // ...
    ctx.send(response);
}

If you need to perform multiple async operations to serve one request then you use future to chain together the async operations, provided that your async operations return a future/promise type. This is not the same as waiting the async operation to return:

void processReq(Req req, Context &ctx) {
    // ...
    op1().then(op2).then(op3);
}

One last word, if you meant waiting for the async code to finish before exiting the process, then use join method of your thread objects, or use the wait method of your promise objects.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.