Where should research and production code reside in git?

Question

We have research code that consists of Jupyter notebooks and large data files. At the same time, we also have production code that consists of Python source and CloudFormation templates. There is currently no overlap between the two - the directory structure is different and research code is rewritten into production code when ready. There might eventually be some overlap though, like a pyproject.toml file but it will certainly be different for both research and production.

One obvious approach is to use separate repositories, perhaps with a naming convention to make it obvious they are related. The problem is that there are many projects so this would effectively double the number of repositories.

Another approach is to use separate branches within a single repository - a research branch that would never be merged and perhaps a dev branch for production that would be merged into main. The problem is that it feels weird to have branches that look totally different and will never be merged.

So, where should research code and production code reside - is there a best practice? For bonus points, considering research code also consists of large data files, is there an approach that would best serve this use case as well?

Answer 1

One good option might be to use two directories within the same branch of the same repository for research and production. Or perhaps if you see production as the main focus use the root directory for production and a subdirectory for research.

Git branches are mostly useful for alternate versions of the same thing, rather than two related things. You switch between branches, but you may well want to be able to look at research and production code side by side. Or if you don't you just close one in your editor/viewer app and open the other.

This will also allow you to trace the history code as its starts life in research and then is potentially deleted from there and added to production (not necessarily in that order).