I write a lot of exploratory code in my research. As I go along, I put functionality that I'd like to reuse in a central location. A project might look like this:
./mylib ./exploration /experiment_1 /experiment_2 /experiment_3
Where each experiment uses some functionality from
Now I come along and start my fourth experiment. In the process, I may need to change my library in some backwards-incompatible way. Now I can't re-run my first three experiments without updating their codes to be compatible with the newest library.
Note: As of now, I keep
mylib in version control, and
exploration in version control, using git. This means that all of the experiments are in the same repository. This is done so that a single
experiments syncs all of my experiments between computers. I feel that there may be a better way, but that might be for another question...
- I could bite the bullet and update the old experiments manually whenever I need to run them (bad, tedious, but straightforward).
- I could "vendorize" my library by copying it whenever I make a new experiment. (Bad, bugfixes have to be inserted into each copy).
- Since I keep my library in version control, I could tag points in the library's history by whatever is required by an experiment. When I want to run experiment n, I'd checkout tag n. (Better, but what if I want to run two experiments simultaneously? It also seems like there should be a way to automatically use a specific version of the library.)
- Whenever I start an experiment, I'll make a new branch in the library. In each experiment folder, I'll clone the library repo and checkout the correct branch. (This seems reasonable, though it is perhaps wasteful of space, since I'm duplicated all branches when I clone. Also, I might have a lot of experiments, meaning that there will be lots and lots of branches in my repository, cluttering things unnecessarily.)
Should I reconsider any of these above solutions?
I have also heard about git's subtrees and submodules, and while they sound like they might be the answer to my problem, I want to get the input of more knowledgeable coders before sinking time into a rabbit-hole.