I have about 60 repos containing Python packages, currently using
setuptools in a setup.py (run via
pip install) to manage third-party dependencies. Most of these packages need to be installed on a system at the same time, which means their dependencies all need to be compatible with eachother.
The dependency versions are specified with
== because I want to lock the dependencies to specific versions, since I've tested the software against those specific versions, and occasionally a new version of a third-party package will introduce a bug.
I'm constantly struggling to keep dependent package versions in sync across all of the repos. For example, if I update somepackage from 1.22.1 to 1.22.2 in one project, I have to then go and modify the setup.py in a few dozen other repos to match.
My partial solution right now is to use a lot of virtual environments, but this isn't a complete solution because I still have to maintain compatible dependencies across all the packages within a given venv.
Also, I'm really trying to move away from having separate venvs for various reasons, including:
- It's inconvenient to have to be constantly switching venvs.
- The entry point scripts that get installed are in the venv's bin, but I'd rather have all applications be installed globally. I can write scripts that switch to the venv then run the application and install those scripts in a global location, but that makes the deployment workflow very clunky.
- If I release an update to one of my packages, I have to install the update in every venv, instead of just once globally. This is inconvenient, error-prone, and time consuming.
My question is: Is there some way to more conveniently manage third-party dependency versions across multiple repos that share some of the same dependencies? Using the same example, if I update somepackage from 1.22.1 to 1.22.2, I'd really like to just change that version number in one place. If it matters, I'd like to support Python back to at least 3.9.
The only possible solution I've been able to come up with is something like:
- Have a package, let's call it mysetup, that exports a bunch of constants containing the version numbers of any dependency that any of the repos use.
- Import that package in setup.py and use it to construct the
But the main problem with that is that mysetup then needs to be manually installed first before installing any of the repo packages, which kind of breaks the deployment flow a bit.
Is there any solution to this?