Let's imagine a custom network application having 3 protocol layers.

enter image description here

Layer A is the top-level protocol and layer C is the lower level protocol. Each layer has its distinct functionalities. These layers need to communicate among them. To my understanding, we can simply call methods of protocol layer B from A's methods.

Recently I got to know that, this layered communication can be implemented as different entities (threads) and they wait on a specific queue to receive messages from other layers. In C++ terms, I can think of using std::queue, std::thread and synchronize read and write to the queue using mutex and conditional variables.

Before implementing this message queue mechanism, I would like to know, is there any abstraction level (key consideration) that I am missing? I don't need any C++ specific implementation details, but if you provide, that's a plus for me.

  • 1
    If you code something up as a POC (Proof of Concept), you should be able to answer your own question. :)
    – Jon Raynor
    Sep 24, 2019 at 14:31
  • What semantics do you need? Should they look like function calls, or can they be asynchronous? Do you need to hide or expose the layering? What should happen if the next layer, or the one after, fails, or is busy?
    – Useless
    Sep 24, 2019 at 15:23
  • Yes, function calls, and they can be asynchronous. If lower layer fails then it lower layer should report to upper layer with some predefined format. Hide or expose is not clear to me.
    – Debashish
    Sep 24, 2019 at 15:31

1 Answer 1


It seems that you have already well thought about your design. I do not see something fundamental missing. But if deemed useful, here some further thoughts.

If I understand well your intent:

  • you’ll have three threads TA, TB, TC
  • each thread will run an instance of a specialized class acting as protocol handler PA, PB, PC
  • the input messages for each protocol handler shall be queued. So you’ll have the queues QA, QB, QC.

It is important to note that a thread does not run an object, but executes code to whatever object it belongs. This is not a problem in your case, since you want to isolate PA, PB and PC from each other, by channelling their interactions through queues instead of direct function calls.

The consequence is that each queue will be touched by several threads. So there is a (high) risk of race condition, when for example one thread writes an element into the queue and at the same time another thread reads the information. Standard library queues are not thread-safe.

To avoid the race you could simply lock a queue when it is accessed. But this might make your queue the bottleneck. Instead, you could consider using a lock free queue to reduce contention (you should use an existing implementation like boost’s one; or the one proposed in Anthony Williams book “C++ concurrency in action”).

Another thing to consider is that you could in addition to the vertical split into threads also split horizontally. So there could be more than one thread dedicated to the same layer. So you could have TA1 and TA2 running PA1 and PA2. All the clones would just have to listen to the same queue QA.

To allow this design and keep it flexible, the reference to the queues to use (input queue, downward and upward output queues) could be provided to the constructor of the protocol handler.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.