Accessing Hardware-based resources from multiple worker threads

Question

I have a technical application that interacts with different hardware components: measurement devices, sensors, custom hardware. These use dedicated interfaces like USB, serial ports, TCP/IP communication and others. Most of these devices' implementations wrap a request/response pattern: Send something to the device and wait for a response to perform a specific task.

The software now has to perform both long and short tasks that all involve communicating with these hardware components. In order to "keep the GUI responsive", I wrapped all long (and even the shorter) tasks in worker threads.

All of that works fine as long as one hardware component is exclusively used by one thread. My trouble begins when e.g. a "background task" uses the same hardware component that is also used by some "long task": Either the related resources cannot be aquired, or the different threads' communications interfere with each other, breaking the request/response pattern of the related resources. Another problem arises when a device has long-lasting init sequences when connecting to it: In these cases it's tempting to just keep a connection to some hardware open, which one the other hand locks out other threads from accessing that hardware longer than needed.

To solve these problems, I have a few solutions in place:

• use a global object that is initialized once, and wrap the request/response part into a mutex that protects from interrupting the request/response sequence.

  Problem here: The mutex part can create problems if there is some kind of event processing between request/response, e.g. with TCP/IP protocols.

• Use a new instance of a hardware object's controller class whenever needing to communicate with that device.

  Problems here:

  • Overhead in initialization/connecting, which is done over and over again
  • must be possible to have simultaneous connections to the hardware or one must wait until hardware can be connected again, which creates quite some overhead and can create subtle bugs.

• wrap all stuff related to a specific hw component in a seperate worker thread, and use that from other worker threads.

  Problems here:

  • additional complexity when connecting workers to each other
  • also needs mutexes or other synchronisation methods with are not obvious from the code interacting with this worker, thus error-prone and hard to debug

I am using Qt / C++ in my software, and typically I use signal/slot for communicating between different threads. However I believe that my use case is not too specific for this framework and programming language, and patterns from other languages might provide nice ideas and solutions ...

My question: What kind of patterns could I use to wrap my hardware components' objects in a consistent way, so I can access them from multiple worker threads while solving my problems described above?

Answer 1

The hardware can only do one thing at a time (you say). Therefore whatever the code is trying to do has to wait until the hardware device is available. No ifs or buts.

You have two easy options to do this: synchronous, or asynchronous.

Synchronous: create a mutex for each device. Before using the device, lock the mutex. After using the device, unlock the mutex. The mutex ensures only one thread can communicate with the device at a time. There's no need to set up anything else. You already identified this option and cited a problem:

Problem here: The mutex part can create problems if there is some kind of event processing between request/response, e.g. with TCP/IP protocols.

So, onto the asynchronous option: Create a worker thread and a blocking queue. C++ does not have one built-in, but Qt does event loops and signal queues for you. When the worker thread gets a signal to send a request, it will send the request, block until it gets the response and then send the response back with another signal. Additional synchronization should not be required, as long as you don't try to access the same data on multiple threads (e.g. changing the request after you sent it).

However, take care about what happens if the response-handling slot gets destroyed before it receives the response. Does Qt detect this situation and ignore the signal? Or does it crash?

When not using Qt, you can still do it similarly but with your own event loop. C++ doesn't have a blocking queue in the standard library, but it's easy enough to make one. You can send the results back using std::promise.

---

This:

Use a new instance of a hardware object's controller class whenever needing to communicate with that device.

is not a solution, it's just one of the other solutions (or non-solutions) with extra steps. The device doesn't care whether you use a new instance or the same instance of your controller class. It also doesn't care whether you use a mutex or multithreading, but it does care that you don't try to send it two requests at the same time and get them mixed up.

Answer 2

1. Analyse which tasks each hardware element can perform, how long each takes, importantly whether the order of tasks makes a difference, and also importantly whether there is manual action needed.

2. Analyse which high level actions you have, and what hardware tasks need to be performed in which order for each high level task.

3. Write software that allows you to enter tasks and submit them for execution, choosing a time and order of operations that gives best results. That’s the hard part. Be prepared that high level tasks may be cancelled completely or partially then resumed, or that they can just fail.

4. Talk to people with more relevant knowledge and listen to them.