Due to the Global Interpreter Lock, multithreading in Python does not affect parallelism. This limitation is avoided in the
multiprocessing library by spawning new processes instead. But threading does improve responsiveness in certain (IO-bound) situations. (from this answer on SO).
My situation is as follows. A Python process is communicated with using
stdout. One line corresponds to one message and is met with a result line. Currently I have a single process consuming the messages one by one and producing a result. However, the processing involves moderately heavy calculations, so better performance could be achieved with parallel computations of different requests.
After reading up on the differences and uses of threading and multiprocessing, I've come up with a structure that would hopefully suit this task.
- Messages ought to be handled asynchronously to allow for reading stdin and distributing work before returning results. Two threads could be spawned to handle the reading and writing.
- Multiple processes could be spawned to respond to those requests. An example from the
multiprocessingdocumentation: here (#3: using Queues)
As the example higlights (with random sleep times), the response order is not fixed. Jobs are returned in the order they finish.
Is this a sensible design? Furthermore, what could be done to preserve the order of messages in the pipeline?