Evaluating concurrent application design approaches on Linux

Question

Working on relatively low-speed hardware, I need to grab a snapshot of a rapidly changing datastream and then post-process the snapshot data and save it to disk.

I need help with designing the overall approach given the processing constraints from the hardware.

---

The snapshot phase needs to complete relatively instantaneously, because it blocks the datastream as it performs the capture. I believe I want to run this as a separate process, or at least as a stand-alone thread. This phase needs to be given priority for processing and accessing resources.

The post-processing phase relies on external libraries that do not use a repeated-small-iteration API/function call model I would be able to throttle the call speed of - instead, a single function call performs the entire operation.

However, I don't care when these post-processing tasks finish, and I would prefer this routine be run in such a way that it vies for very little CPU/IO while working. My thinking is that this would encourage future snapshot events to complete with a minimum of latency. The post-processing only takes a few seconds to complete, but I far prioritize snapshot latency over post-processing speed.

Taking all of these requirements into account, it makes sense to me to run both of these actions as separate, concurrent tasks, and tell the kernel the snapshot task is extremely high priority, and the post-process task is extremely low priority.

I generically say task here, and I said "concurrent" in the title, because I'm not sure whether I want to use separate processes for this, or implement some form of multi-threading.
I have only a little experience with fork() and the like, and no real experience with threading, so I have a few questions about how/where I should get started:

• Which method would be preferable - threading or separate processes? Does utilizing threads simplify sharing memory access between the two tasks?

• Can I set kernel scheduling/IO priority when using threads? Or am I worrying too much about setting kernel priority at this point?

• To keep the snapshot process fast, it would be ideal for the snapshot task to simply leave data to process sitting in memory for the post-process task to pick up and run with, so the snapshot task is immediately clear and waiting for the next event. This seems to be best addressed with a queuing approach, but I don't know if there are approaches that provide other advantages.

If I was using fork(), I could simply leave unprocessed data in a pipe() for the post-process task to clean up after. I'm confident that the pipe wouldn't become full and block the snapshot process.
I'm not as certain what I can do with threading. Could I possibly allocate memory from the snapshot thread; share the pointers via IPC; then free() the data from the post-process thread? Do I gain any processing speed or other benefits by taking this approach?

---

To help focus my question, I have two broad paths I'm considering.

1. Two separate processes using the OS's pipe() functionality to provide communication between them.

2. A multi-threaded process using IPC or some other communication mechanism.

I need to be able to prioritize one task over the other, and possibly over other processes running on the system. I have time constraints for executing one of the tasks, but I don't care about execution time for the second task.

Are my two potential paths equivalent, or is there a preferred approach given my constraints? Or is there a third approach that is better than those two?

Answer 1

Which method would be preferable - threading or separate processes? Does utilizing threads simplify sharing memory access between the two tasks?

Threads do simplify (and greatly improve the performance of) shared memory access. You would use processes if you wanted to make sure that in the event that one of the tasks crashes the other task remains unaffected, but in your particular scenario, that would not buy you anything: if the snapshot task crashes, the post-processing task has no work to do, so it is moot point if it survives the crash. On the other hand, if the post-processing task crashes, the raw data obtained by the snapshot task will be lost. So, unless you are willing to temporarily buffer the raw data on the disk, two processes are not needed; two threads will work just fine.

Can I set kernel scheduling/IO priority when using threads? Or am I worrying too much about setting kernel priority at this point?

Yes, can do it with threads. And you probably won't even have to go to extremes: the snapshot task only needs to have a slightly higher priority than the the post-processing task in order to make sure that the post-processing task never preempts the snapshot task. However, if you want to make sure that nothing else on the machine delays the snapshot task, use real-time (max) priority for the snapshot task, and normal priority for the post-processing task.

To keep the snapshot process fast, it would be ideal for the snapshot task to simply leave data to process sitting in memory for the post-process task to pick up and run with, so the snapshot task is immediately clear and waiting for the next event. This seems to be best addressed with a queuing approach, but I don't know if there are approaches that provide other advantages.

Yes, queuing will work just fine. You might want to make use not of a queue of data, but of a queue of pointers to memory blocks containing data, so as to minimize copying of data around.

Now, all that having been said, you should examine the possibility of using the disk as a buffer between your two tasks, because this way, if you ever have to temporarily stop the post-processing task, you will not lose the data generated by the snapshot task while the post-processing task is stopped. This will in turn enable you to also use separate processes for the two tasks, which keeps things clean, and provides additional safety: if one of the tasks crashes, the other can continue working. If the post-processing task crashes, the snapshot task can continue generating raw data which will not be lost because it is being persisted, and if the snapshot task crashes, the post-processing task may have a backlog of previously persisted raw data to keep itself busy with.

If you are sure you will never need to persist raw data to the disk, then another issue becomes evident: neither multi-threading nor multi-tasking buys you anything!

You see,the post-processing task will never be allowed to preempt the snapshot task, but it will always have to be consuming raw data at the same or higher rate than the rate at which the snapshot task produces raw data, or else data will be lost. So, this means that you can have both "tasks" work in lockstep fashion, in the same thread: first the snapshot task is invoked to generate data, then the post-processing task is invoked to process the data, then you may or may not wait until it is time to take another snapshot, and repeat.