We're developing some C++ application that analyses strings using some data. The data is large (a few gigabytes) and takes a significant amount of time to load into memory, and timing, as well as stability is crucial for the application.
We're currently discussing the option of splitting our monolithic application into some kind of a "master/slave" relationship, where the "master" would get a string to process, run it through various processes, and return the information back to the user.
Each "slave" would be an executable that is being called and it communicates back to the "master" when it's done.
The "master" would act as a daemon and run in the background continuously, listening to new requests and processes them accordingly. It would also be in charge of initially loading the data into a shared memory space where the rest of the executables would get their information from (read only) and manipulate the input string accordingly.
The data regarding the specific string being processed would be written into an another shared space, or possibly a memory-mapped file, and as it's being processed linearly by each "slave", so write protections shouldn't be really a concern, since for each job there would be only 1 slave accessing it at any time.
Other than harder debugging of the shared memory stuff, what are the problems I am overlooking with this approach?