Splitting a critical application into multiple executables

Question

We're developing some C++ application that analyses strings using some data. The data is large (a few gigabytes) and takes a significant amount of time to load into memory, and timing, as well as stability is crucial for the application.

We're currently discussing the option of splitting our monolithic application into some kind of a "master/slave" relationship, where the "master" would get a string to process, run it through various processes, and return the information back to the user.

Each "slave" would be an executable that is being called and it communicates back to the "master" when it's done.

The "master" would act as a daemon and run in the background continuously, listening to new requests and processes them accordingly. It would also be in charge of initially loading the data into a shared memory space where the rest of the executables would get their information from (read only) and manipulate the input string accordingly.

The data regarding the specific string being processed would be written into an another shared space, or possibly a memory-mapped file, and as it's being processed linearly by each "slave", so write protections shouldn't be really a concern, since for each job there would be only 1 slave accessing it at any time.

Other than harder debugging of the shared memory stuff, what are the problems I am overlooking with this approach?

Answer 1

It seems based on the tags you're interested in two aspects:

• Memory
• Performance

I'd say that if the operations the slaves do are pretty self-contained and there is not much communication between master and slaves, the performance aspect will be fulfilled by your master/slave approach. Obviously you'll need to feed all of the data to a slave, but then again the master will have to get the data somehow, and master-slave communication isn't slower than master-datasource communication.

About memory: if the slaves have to allocate lots of memory that will be subsequently freed, it actually might be beneficial to terminate the entire slave process after the operation. If you perform everything in the master, the heap will eventually become fragmented. It may also be the case that the malloc() library can't free all memory back to the operating system, because there is some tiny bit of memory in use inside a massive arena malloc() has mmap()'d.

Also, as you noticed, C++ isn't a safe language and crashes do happen. By having less operations in one application, it might be simpler to investigate core dumps. It's also better design to isolate crashes to a smaller application than to have a huge monolithic application.

Go for it! The Unix philosophy is that one program does only one thing, but does it well. It's a good philosophy. Recently, another term has emerged for this kind of architecture: https://en.wikipedia.org/wiki/Microservices

Answer 2

I would suggest looking first for ways how to prepare large data to be crushed in an efficient way. You should look into various techniques or tools that you can find on the internet. If you are doing everything from scratch you might need to return to the design and look into:

• How the data is being parsed, maybe you need to segment and create a thread pipeline for each segment; (I do not see the problem if you are dealing with strings);
• Try to reuse the same memory block, meaning that you will need to create a structure; If you to go a little deeper you can try to create your own class template.
• The concept of Master/Slave is more for used redundancy, what you want to say is a kind of launching child process.