2

I have a function that performs a task which can either be performed immediately and without any blocking or it can only be performed after some delay. The caller of that function shouldn't need to care, that's why this function has a completion handler and will perform the task asynchronously if required and then call the completion handler once the task completed.

The current pseudo implementation is like this:

func doSomething ( arg, completion ) 
{
    if (doSomethingInternal(arg)) {
        completion()
    } else {
        scheduleForLater(arg, completion)
    }
}

If doSomethingInternal() returns true, the task was performed, otherwise it wasn't performed and we schedule it for later.

Now here's the catch:

As you can see, if the task can be performed at once, the completion handler is called on the same thread that called doSomething() and it is called even before doSomething() has returned. In case it must be scheduled for later, the completion handler is called on a different thread and it may be called on that thread even before doSomething() has returned or after it has returned (the later one is much more likely).

I considered this kind of implementation to be a good idea as it avoids an unnecessary thread switch and thus has a low overhead as long as the task can be performed instantly which is the case > 90% of the time. Yet when putting it to use, I ran into a problem, as demonstrated by the following (very simplified) pseudo code:

func performNextTask ( ) 
{
    arg = getNextArg()
    if (arg) doSomething(arg, { performNextTask() })
}

As getNextArg() is thread-safe, it won't matter on what thread performNextTask() is getting called and if I force doSomething() to always be asynchronous, the entire code works correctly. However, if it performs synchronously all the time, the code above will crash as it runs into a stack overflow! That's because in that case you get a recursive call chain if there's always a next arg available.

So here's my design question: How do you usually deal with that situation?

Never be synchronous? Will work but result in sub-optimal performance because of unnecessary thread switches.

Let the completion know, if it is getting called synchronously or not? That way the completion itself can somehow deal with that situation, yet I have never seen any code doing that.

Not call the completion in the synchronous case and let the caller know that the completion won't be called as the task was already performed? I have seen that in the wild but to me that is code smell as many programmers will expect a completion to always be called when something has completed.

Any other ideas?

0
2

Yes, thread-switching is pretty expensive.

But deferring the completion by queuing it for later does not in any way imply the need for switching threads.
It's just the cost of queuing it up, retrieving it later, and the tasks data potentially no longer being quite as hot.

The more important point is the slight but potentially significant change in semantics:
The caller must ensure that the completion callback being called immediately or deferred is equivalent, limiting the caller. That can be quite involved, or trivial, have a huge runtime cost, or be essentially free.

Anyway, it is an additional concern that must not be neglected.

7
  • In my case the delayed callback will always happen on a different thread as I have no way to perform it on the thread of the original caller (I have nor reference to it nor could I make it do stuff even if I had) and even if I could do that, it would involve another thread switch as the event telling me that I can now doSomethingInternal() without blocking will never happen on that thread.
    – Mecki
    Dec 21 '21 at 9:05
  • As for the "The caller must ensure...", even if the callback is always performed on a background thread, the caller must always be prepared that it happens prior to return of doSomething(), as the call thread may be interrupted by the thread scheduler prior to return, the background thread may get runtime and then perform the callback.
    – Mecki
    Dec 21 '21 at 9:07
  • @Mecki The point was that being threadsafe isn't enough. Remember that simply waiting on yourself to have done something will take forever. Dec 21 '21 at 13:15
  • Not sure what you mean by "waiting on yourself". E.g. if you create a semaphore with value 0 before calling doSomething(), signal the semaphore in the completion and wait on the semaphore after doSomething(), this will correctly turn the async call into a sync call, regardless if completion is called on another thread or on the same one, as either the semaphore was already signaled when the call returns, in which case wait won't block at all, or the current thread will block until the semaphore is signaled on a background thread.
    – Mecki
    Dec 21 '21 at 13:23
  • Consider the caller digs deep into some data-structure, locking at least for reading. If the completion is deferred, updating that datastructure is no problem. If it is immediately executed, trying to do so will wait on the caller to release it, which it won't until the after the callee returned, thus waiting forever. Dec 21 '21 at 13:28
2

An API that sometimes invokes a callback synchronously and sometimes defers its execution can be difficult for the caller to reason about.

If the caller supposedly doesn’t care whether the callback is invoked immediately or not, then that suggests to me that the cost of deferring it isn’t a problem and you may be better off always deferring it.

4
  • To answer your question: The caller surely does care but if there is no data to return, there is no data to return and in that case the caller wants to get new data ASAP without having to block for it. If the caller has to wait for data, the thread switch is neglectable and any alternative way of retrieving it would be slower to begin with, but if data is available right now (the 90% case), it isn't neglectable.
    – Mecki
    Dec 26 '21 at 1:58
  • And unfortunately I cannot know in advance if data is available. It would be great if I could but I haven't designed the API I have to call to figure it out. This API only allows me to retrieve the data right now or telling me there is none. In the later case I can register for a callback once new data has arrived and this registration alone is very costly as my profiler told me. So you see, the preconditions suck already and I have to design a middleware that hides all this ugly low level stuff from the rest of the app and is as efficient as it can be.
    – Mecki
    Dec 26 '21 at 2:02
  • But your linked article is very interesting and the article it links to is even more interesting. I will surely read both of them in all detail. One thing I already learned and considered in my code is that no matter if sync or async, at no time a callback is made while any locks are still held. Everything about my code is perfectly re-entrant save.
    – Mecki
    Dec 26 '21 at 2:04
  • There is a difference between not caring whether it sometimes will be deferred, and accepting it as sometimes necessary. The performance-advantage of only deferring when unavoidable might be worth it. Jan 8 at 21:10
-1

Performing a callback on the same thread is not an anti pattern. But in your case, look at what calls will be performed if there are 10 arguments. It’s too “interesting” for my taste.

2
  • Not sure what you mean by "10 arguments"? The calls would look the same, just passing more arguments along.
    – Mecki
    Dec 21 '21 at 9:03
  • On the same thread is not the question. Opportunistically synchronously is. Jan 20 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.