I need some advice on my idea to write a controller file in Python, and not like the C in MVC type controller but a more simpler idea. Just a Python script that controls the operation of some other Python scripts.
I will outline the idea, and any advice or ideas would be greatly appreciated.
I have to write a system that does some background tasks for a web application. The web app is in PHP and separate from the Python system I am writing, the Python will do some automated tasks and calculate and save some analytical data for the web app to use and display.
I have experience with basic Python but I have not pushed forward to learn much about class/objective Python. So far what I have written for the system is a bunch of separate py scripts that do the jobs they have to do, written totally procedurally. They all work fine, but it's not really a system yet. My initial krux was that I had to keep them all alive and running infinitely in the background. After some research I decided that the best way to do this was to have a process manager like Supervisord look after each script, I would start up the scripts and allow them to run for a certain time with a finite loop (in each script) and then they would exit, then the supervisor would restart them every time they exit and keep them alive. I wanted to avoid using infinite loops because of the possibility of memory leaks (I believe allowing the process to exit totally frees up memory associated with that process and prevents leaks). I have not tested any of this, it's just a concept at the moment.
Something I would prefer to do would be to have one script be the controller of the rest of the scripts and have the supervisor only look after the controller. This saves me having to update the supervisors config file every time I add to the Python system.
The basic idea for the controller would be to have a loop fork new threads for each separate script/process and then have the controller wait for all the other threads to complete before allowing the loop to do another iteration (starting everything again).
I'm guessing what I need to do for this would be to wrap all the code in each script in a function, then import them as modules into the controller and call each function on a new thread inside the 'main' loop of the controller. Then the loop just runs, calls each function on a new thread and waits for each thread to finish before starting again.
How does this sound as a concept?
The only direct coding question I have is about turning my scripts into modules, how do I go from totally procedural code (no functions, no classes) into an importable module? can I wrap the procedures in just a function, or do I need to have them in a class also?