We recently migrated from SVN, with most code in a single repo, to git, with most projects in their own repos (about 70 of them). We build about a dozen different apps from this java source. The apps all run on *nix servers. We use maven and nexus to build. Many of us are struggling with developing features when that feature touches more than one repo. Here are a few of the challenges:

  • The developer has to branch each repo separately - we use the same name for all branches for one feature to make tracking less difficult.

  • One must update poms of all repos to point to the updated versions of each repo's artifact. If multiple people are working on the same branch, there can be a lot of merging others pom changes. When I commit a change to a repo, then the artifact is renamed to "-SNAPSHOT" which means more pom updates.

  • Changes need to be pushed in the right order or our automated builds will fail, e.g: repo A depends on a change to repo B; if repo A is pushed before repo B is built and deployed, then repo A won't build.

  • The person reviewing the feature has to look at changes in multiple repos.

  • When the feature is merged from its branch to, say, master, One has to remember all the repos that were touched.

It looks like switching to a mostly monorepo approach might be best, tho there are some drawbacks there:

  • Building the entire codebase with maven takes a looong time. (Why can't maven be more like make, only building things that have changed or whose dependencies have changed?)

  • Each push kicks off a big set of builds and many unit tests rather than just one repo's artifact build and test.

  • The developers who generally work in one or two repos prefer this new multi-repo world and will resist a change back.

I've looked into git submodules and sub trees, which don't seem to solve many of our issues (Not sure about Google Repo). Some of us use tools like "mu" to help. It would be sweet if there was a toolkit that would help developers maintain versions in poms, and track changes across repos.

Let me know if you have a set of procedures or tools you use to ease development in this kind of environment.

  • I think a reason behind the push for having multiple repositories exists, though you may not agree. A new system may be awkward but necessary. Try to automate all the policies you can and for the ones you can't, make sure everyone knows the proper procedure. Having a single repository in my experience tends to result in every project depending on every other project. Not ever a good idea. – Neil Oct 11 at 8:21
  • 1
    One of the teams I work with recently tried this unsuccessfully (my team is managing it nicely). The problem for them was that their repos weren't sufficiently modular, and frequently they would need to modify a large number of them even a simple change. Basically, before you go multi-repo, you should go fully modular in a mono-repo, so that when you break up into a multi-repo, your code is nicely modular and changes typically only affect a small number of repos. – Maybe_Factor Oct 12 at 5:28

It might not be a direct answer, but I have the feeling these are more the symptoms than the "disease".

If I may, I think the main issue is that a feature implies touching so many different projects. I know we don't live in an ideal world, but perhaps putting aside the technical details and increasing the "low coupling / high cohesion" of your different projects may be an interesting goal.

If several projects are so tighly coupled that they are always touched together, put them together in a common repo. If a big chunk of code can be extracted as relatively independent library, do so.

Perhaps the most pragmatic approach is neither "each small project in a repo" nor "a giant monorepo with everything" but something in-between where closely related projects are bundled together in a repo, while relatively independent ones stay in their repos.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.