My project has a single message stream (communicating with an external device) which is accessed via multiple client threads. The workflow is:
- There is a thread (the "messaging thread") which reads the message stream and is able to dispatch received messages to clients. This works by waiting on a thread-safe queue which the message stream exposes.
- Messages may be sent on the stream from any thread (the stream is thread-safe and handles concurrent send/receive)
- A client thread sends a request across the message stream, and provides a callback function.
- The client thread waits for responses. There may be multiple responses. The messaging thread knows how to match a received message to a client thread, based on the request sent by the client thread. The messaging thread invokes the right callback function for each received message.
- After sending the request, the client thread must block until all responses have been processed. This is normally indicated by the callback function returning a particular status, but it must also be possible to unblock a client thread and end the interaction by the messaging server receiving a particular message which it will recognize, or the application shutting down.
- From time to time the messaging service may be unavailable (the messaging thread will know this); so the client threads must be able to wait for a certain time to see if the messaging service has become available.
Those are my requirements but I can't get my head around how to code the logic for the client threads "entering" and waiting; and how the client threads should communicate with the messaging thread. Which synchronization primitives to use, and so on.
I'm coding in C++ with the Poco libraries, and can use all the usual primitives (mutex, event, semaphore, condition variable etc.) as well as higher level constructs like a notification queue, notification centre, and event dispatcher.