I've been using python for several years as a non-developer writing scripts to help business processes. I'm in a new position (still non-developer) and would like to adopt more rigorous coding standards (especially since the projects I will be working on will be bigger). Specifically, this means, in part, creating distinct environments and folder structures for each project rather than just having a bunch of modules from distinct projects all floating together in the same folder getting run manually at the terminal.
I was reading this guide about how to structure projects but I have some questions since I've never done this before.
1) An example project I am tasked with includes a. automating data import into system b. checking downstream output from system to ensure everything was processed correctly.
Data takes at least 1 month to flow through, this is not an immediate process. In other words, these are distinct tasks. But higher ups have phrased it as being one project. In the world of structuring python projects, would it make more sense to create two projects (or in other words, two folder structures with their own virtual environments) out of this? Or include them in one? I don't foresee code needing to be shared between them. I do imagine task b. would need a list of case IDs created by task a. but that would be the only functional link.
I assume two distinct projects makes sense but what thought goes into that?
2) Is a project designed to only ever explicitly run a single module (like a main.py at the top level) while the others are all referenced internally? Or can you structure a project such that you may explicitly run any number of modules depending on what task you are performing at runtime? In other words, what is the scope and intent of a project/top-level package/folder?
3) If you intend to run a main.py from the top level package at the command line, does that top level directory need to be a package if it is not being imported into another program? Can it just be a regular folder as long as you never reference it in other code?
4) To what degree are you "supposed" to combine classes/functions with the nitty-gritty usage of those functions (is there a term for this?) within the same module? In other words, are you supposed to leave classes/functions in their own modules and leave the importing/usage/running of them to another separate module? Or can you combine them into one? And when would you do which and why?
With my messy background, I have only ever had singular modules which contain all the functions and execution in a single file. I don't know what the proper protocol is for delimiting scope.
These may be basic questions that I've taken too long to come around to but thank you.