We have research code that consists of Jupyter notebooks and large data files. At the same time, we also have production code that consists of Python source and CloudFormation templates. There is currently no overlap between the two - the directory structure is different and research code is rewritten into production code when ready. There might eventually be some overlap though, like a
pyproject.toml file but it will certainly be different for both research and production.
One obvious approach is to use separate repositories, perhaps with a naming convention to make it obvious they are related. The problem is that there are many projects so this would effectively double the number of repositories.
Another approach is to use separate branches within a single repository - a
research branch that would never be merged and perhaps a
dev branch for production that would be merged into
main. The problem is that it feels weird to have branches that look totally different and will never be merged.
So, where should research code and production code reside - is there a best practice? For bonus points, considering research code also consists of large data files, is there an approach that would best serve this use case as well?