I am working on the design of an application that consists of three parts:
- a single thread that watches for certain events happening (file creation, external requests etc.)
- N worker threads that respond to these events by processing them (each worker processes and consumes a single event and the processing can take variable time)
- a controller that manages those threads and does error handling (restarting of threads, logging of results)
Although this is pretty basic and not difficult to implement, I am wondering what would be the "right" way to do it (in this concrete case in Java, but higher abstraction answers are also appreciated). Two strategies come to mind:
Observer/Observable: The watching thread is observed by the controller. In case of an event happening, the controller is then notified and can assign the new task to a free thread from a reusable cached thread pool (or wait and cache the tasks in FIFO queue if all threads are currently busy). The worker threads implement Callable and either return successfull with the result (or a boolean value), or return with an error, in which case the controller may decide what to to (depending on the nature of error that has happended).
Producer/Consumer: The watching thread shares a BlockingQueue with the controller (event-queue) and the controller shares two with all workers (task-queue and result-queue). In case of an event, the watching thread puts a task object in the event-queue. The controller takes new tasks from the event-queue, reviews them and puts them in the task-queue. Each worker waits for new tasks and takes/consumes them from the task-queue (first come first served, managed by the queue itself), putting the results or errors back into the result-queue. Finally, the controller can retrieve the results from the result-queue and take according steps in case of errors.
The end results of both approaches are similar, but they each have slight differences:
With Observers, the control of threads is direct and each task is attributed to a specific new spawned worker. Overhead for creation of threads may be higher, but not much thanks to the cached thread pool. On the other hand, the Observer pattern is reduced to a single Observer instead of multiple, which is not exactly what it was designed for.
The queue strategy seems to be easier to extend, for example adding multiple producers instead of one is straightforward and does not require any change. The downside is that all threads would run indefinitely, even when not doing any work at all, and error/result handling does not look as elegant as in the first solution.
What would be the most fitting approach in this situation and why? I have found it difficult to find answers to this question online, because most examples only deal with clear cases, like updating many windows with a new value in the Observer case or processing with multiple consumers and producers. Any input is greatly appreciated.