I have been debating this question internally for a few weeks now and I keep coming up short in finding a good answer to the question. I feel an example would be more descriptive than just talking about it so I will start with that.
I have a Python project repository structured as follows:
. ├── common_package │ ├── __init__.py │ └── other files and folders... │ ├── package_one | ├── src │ | ├── common_package -> ../common_package │ │ └── package_one │ │ ├── __init__.py │ │ └── other files and folders... │ ├── tests │ │ └── ... │ ├── README.md │ ├── setup.cfg │ └── setup.py │ ├── package_two | ├── src │ | ├── common_package -> ../common_package │ │ └── package_two │ │ ├── __init__.py │ │ └── other files and folders... │ ├── tests │ │ └── ... │ ├── README.md │ ├── setup.cfg │ └── setup.py │ └── other packages with common_package...
Each one of
package_two, etc... is a separate Python package with its own (wildly varying and fairly large) dependencies, build process, and distributions. However, all of these packages implement (in some form or another) things in
common_package which provides a common interface to the different packages for competition in the future. The packages are completely disjoint other than that (barring a resources system managed by
Some packages are fairly small (few files each) so they do not warrant having a separate repository for each; also since their dependencies vary quite a bit and the techniques are very disjoint (and there is no use for any package in any other package), I do not think it would be wise to package them all under one
I found this technique to be quite useful and practical, with one caveat: after installing a package, I now have
common_package as an accessible, importable package from anywhere in my virtual environment. I fear this spells some bad news since the package name is very generic (in my implementation it is just
common), and it could be overridden.
For example, this is how some code would look after I install
>>> from common_package import config >>> from package_one import some_module >>> some_module.do_stuff(config.SOME_PATH)
So my question is the following: is there any way I can improve on this structure? Particularly, can I someone encapsulate all of these sub-packages under one "namespace", such as
project_name without having them be under one package that has to all be installed at the same time?