How to convert multiple Git repositories for related code into proper branches/tags

Question

Background

I've recently become the technical lead/senior developer for a small R&D/data science team. I've got a decent amount of non-lead experience in bigger development teams, but I've not got any formal education in project or team management.

There are a few experienced programmers on the team, but the others haven't had much experience working on bigger projects. They currently use Git to manage their codebase albeit in a rather unconventional way: So far, people have been effectively "branching" by copying the entire codebase for a project as a new repository. They effectively "tag" the repository with a descriptive name and then largely don't touch it once they "release" the tagged code unless some critical bug is discovered.

For example, here is how this would look for the products named arthur and lancelot:

small_companys_code
├── arthur (this is the "basic" bleeding edge "branch" of project "arthur")
├── arthur-Jan2015 (this is a "tag" for a stable release in Jan 2015)
├── arthur-multilang (this is a "development branch" for the new feature "multilang")
├── arthur-multilang-Mar2016 (this is a "tag" for a release in Mar 2016 with a new feature "multilang", maybe not necessarily merged into the repo "arthur")
├── arthur-fred (these are the personal changes of the former employee Fred. No one remembers exactly what changes this has)
├── lancelot (this is the "basic" bleeding edge "branch" of project "lancelot")
├── lancelot-Feb2018 (this is a "tag" for a stable release in Feb 2018)
├── lancelot-Feb2018-spanish (this is almost exactly the same as "lancelot-Feb2018" but was quickly changed to support Spanish for demonstrating)

This works well enough for the people who've stayed long enough to know where everything is. However, for new people (myself included), this is likely very daunting.

Goal

One of my official responsibilities is to "clean this up" and in doing so primarily ensure that there is always a working, releasable version of each project (arthur and lancelot above). The exact workflow is undecided at the moment; My immediate task is to re-organize the code primarily so that there is always a releasable "latest" stable version of the code which can be easily deployed and run even if it doesn't necessarily have all the features developed for other versions (i.e. experimental or features planned for the future):

small_companys_code
├── arthur (repo for project "arthur")
|   ├── master (branch for project "arthur" including only tested features which will definitely be included in later versions of product)
|   |   ├── Jan2015 (release tag)
|   |   ├── multilang (branch for new feature)
|   |   |   ├── Mar2016 (release tag for demo'ed version of feature "multilang")
|   |   ├── fred (orphaned branch to eventually be merged/removed)
├── lancelot (repo for project "lancelot")
|   ├── master (branch for project "lancelot" including only tested features which will definitely be included in later versions of product)
|   |   ├── Feb2018 (release tag)
|   |   ├── spanish (branch for quickly-hacked "feature", possibly to be developed or merged but maybe just removed)
|   |   |   ├──  Feb2018 (release tag for one-off demo)

How can I approach this task? Since the repositories were essentially copied in most cases, I can't rely on searching through the Git commit logs. I can of course look at the actual real-world dates for the changes and create a chronological ordering of the different repositories, but I'm not sure how I can use this information to create a proper Git workflow like the one shown above.

Answer 1

Since these individual repositories were copied from the original, you can re-establish the relationship between the copies and the original. See this other answer for more details.

Use git remote add in each of the copies to reference the original. You can then ensure that the branch names in each copy don't conflict between copies, and start re-synchronizing the copies to the original. Git will take care of re-establishing the tree structure in the original repository, because that is what git does.

Answer 2

Science teams can be tricky to work with. They typically have little to no background in software engineering, and software is production-ready when it works.

It sounds like you've got the technical solution from @BobDalgleish - which sounds right to me. But you've got a larger mental model problem that has pushed people working in software towards a solution that is inappropriate. Copying the entire repo, changing it slightly, and then releasing it for Project x/y/z shows a lack of foresight.

Perhaps a more appropriate solution is to create a plan to adjust the way people work - change them from making a short-term goal work the fastest way possible, to a longer-term goal of increasing the feature set available so that there will be projects that require no custom code. The idea of what a true release is may be foreign to them and you'll need to take the lead.

Try not to tell them they've been working wrong and you're going to correct them - the projects must have been successful on some level. Try to get them bought into the idea of the bigger picture and then help them turn the corner by doing a lot of work and structuring and helping them deliver smaller projects within the framework.