5

I'm considering creating a new tool, but I wanted to find out if something like it already exists.

tl;dr: Looking to make or employ a service which scans a SCM repository with the explicit goal of early warning when ongoing work might lead to "Conflict hell".

The Pain:

Working with many collaborators on a monorepo, conflicts are not possible to detect until specific effort is made to merge together work.

Whether it is the master/main/develop branch that I'm conflicting with or another teammate's feature branch, I must take a premeditated action to attempt to merge in or rebase on the other changes in order to discover that a conflict exists.

The Traditional Solution:

Communicate closely and regularly with teammates to limit the amount of overlap. Just work on something else, if someone might be currently touching or about to touch that code.

The Proposed Solution:

A service that monitors the git repo and proactively determines conflicts between branches. Mechanically, this can be done by checking out one branch and attempting a merge or patch on the other or any other suitable way. Feature branches are associated with "owners". The master branch is a special case which has no owner. Heuristics are put in place to mark branches as inactive if their recent activity is older than some amount of time, and also if they have been squashed into master, so that only branches actively being worked on are checked. The number of conflict checks thus performed is quadratic on the number of active branches, so it should be a manageable amount of processing for a typical team size contributing to one repo.

What this service would provide, then, is a proactive and timely notification to the only people for whom the information is relevant: "Your work and your colleague's work are now currently conflicting, and here is the conflicting content". This will serve to catch the situations when your pre-planning has failed, but also it can serve as a supplement to planning. It can provide tactical feedback on how much toe-stepping has happened, allowing for timely correction.

One requirement for this to work well would be that each contributor has to commit and push regularly in order to benefit from this service. But that's short-selling it because it should still be plenty of good forwarning if you have a long running branch and a colleague all of a sudden pushes and merges a big change into master, getting notified at this point is still better than finding out about it only when you complete your work and make your own PR.

Another point I think is that if this works as well as I hope it could work, then it's possible that some aspects of development planning can be streamlined, saving some time and overhead from the development process. I would draw a parallel here to the Just-In-Time concept. Here, when you step on somebody's toes, it's caught (after the fact, unfortunately), but with a much smaller delay than usual, the delay is roughly until your next commit and push. I think if the concept pans out, it should also be possible to reduce the delay to move closer to a true Just-In-Time experience, editor integration will be required, though. Taken to its logical conclusion, an entire real-time collaboration layer would exist above git, integrated into the editor.

Does anyone else see the value of such a tool? I'm not seeing any downsides whatsoever, this can run as an independent service, and fire off emails based on readily available git user information. It would be zero-effort to set up from the perspective of participating team members. If this idea has merit, does such a tool already exist, so I could try it out? Or should I just try to make it? Thanks.

9
  • 10
    Merge often. Don't use long-lived feature branches. Use trunk-based development: all devs should merge into trunk at least once a day. Preferably more often. Architect your software to allow incomplete features to be turned off. Employ heavy test automation to ensure merges don't break existing code and won't block other's work.
    – Euphoric
    Commented Jul 12, 2020 at 8:56
  • @Euphoric I 100% agree with you that trunk-based development would eliminate this pain and I agree that having more flexibility in features and having heavy test automation will reduce the pain. But I feel like you're just coming at this from the perspective of a well-established product that is perhaps not constantly undergoing architectural changes. The topic that I want to discuss is (as you show) probably irrelevant in the case of an already-mature software project.
    – Steven Lu
    Commented Jul 12, 2020 at 9:43
  • 1
    There are scenarios where you cannot just keep merging in a broken feature every night. Every project is different. It can become counterproductive really quick! Suppose I'm 1/4 of the way through my refactor of a C++ codebase and now it's the end of the day, well, just getting to a spot where the code even compiles will add another hour of work! But it's not just that, since it is a refactor, sticking to the incremental development happy path could be out of the question. In the real world your branches live longer than you want them to. It is entirely valid to have long-lived branches.
    – Steven Lu
    Commented Jul 12, 2020 at 9:51
  • 2
    The answer to that is to not set out on a four day refactor of the whole codebase to get to a single mergeable point - identify and apply incremental changes, e.g. using the "parallel change" pattern. This reduces the risk of other changes in the meantime breaking what you've done, or having to tell the team you can't work on a new, high-priority item because you still have days left on the refactor. You don't need a mature, well-established product to practice evolutionary architecture (indeed by that point you likely don't need to!)
    – jonrsharpe
    Commented Jul 12, 2020 at 10:54
  • 2
    Refactoring, by definition, is taking small steps, each taking few minutes at most, with code being in working state between each of those steps. If your codebase is not working for 4 days, then you are not doing refactoring.
    – Euphoric
    Commented Jul 12, 2020 at 11:51

2 Answers 2

1

"Your work and your colleague's work are now currently conflicting, and here is the conflicting content".

Notifications should be actionable. If they're not actionable, they start feeling spammy. When I get a message like this, what could/should I do about it? Rebase?

Does this reduce the overall amount of manual labour dealing with merge conflicts, or does it just spread it out rather than deferring it to the last minute before integration?

How does this interact with various git workflows and review processes? At my current job we use Gerrit, which is focused on "patch sets" rather than branches. It automatically shows you in its web UI when a patch set is not mergeable.

2
  • > what could/should I do about it? Rebase? Yes. It would be able to notify me if I had decided to start working on something that a colleague has already started touching. That may be extremely valuable, allow me to act on it via communicating with the colleague or abandoning the (early) work I started. I use Github both for contributing to open source projects and at work. It of course shows a conflict when a PR is created. The point of this is that it's not early enough.
    – Steven Lu
    Commented Aug 12, 2020 at 11:22
  • Yes it is actionable. I don’t understand why you ask when you answer with one option: rebase (another is merge). And yes, doing it proactively makes things a lot more simple rather than waiting a long time, letting conflicts pile up. Unless you use a piecemeal merge tool like git-imerge Commented Nov 29 at 16:58
0

The "traditional solution" is to have a repo per project.

You are going to have to go further than just checking for file conflicts. You will have to build and test the merged code to have meaningfull results.

Otherwise you miss any conflict that isnt a simple dual file edit and get a lot of false positives for compatible file edits.

If you read through what mono repo proponents are doing, its not having one git repo with all the code in it. It's trunk based micro commits with CI that understands the project boundaries within the code base. When they talk about branches, its perforce style branching rather than git style.

10
  • Thanks, could you expand a bit on perforce-style branching as opposed to git-style branching? I'm not familiar on what the difference is supposed to be here. I've also used perforce in the past.
    – Steven Lu
    Commented Jul 12, 2020 at 13:49
  • I guess its best summed up by saying branching is a bigger deal in perforce. You can only merge up the tree, it allows locking and the branches are centralised. In Git you can merge and branch more freely and indeed are forced to have local branches.
    – Ewan
    Commented Jul 12, 2020 at 15:53
  • I think you misunderstood the premise of my question. This is not a question about how to lay out a repo... the idea is that conflict/merge hell is a real-world factor that if you are a dev you know can severely affect the velocity of development, and I want a tool that can attempt to provide early-warning signals to developers. It matters not whether code is being worked on in separate repos. This tool would need to use heuristics to know what code is "actively being worked on", and then it will check if the changes would merge together cleanly. If not, alert devs.
    – Steven Lu
    Commented Dec 8, 2021 at 20:36
  • I realize that maybe you got to thinking because I did specifically use the word "monorepo" in my question. However this would appear to be a total red herring
    – Steven Lu
    Commented Dec 8, 2021 at 20:39
  • The tool you describe is goes hand in hand with the concept of a monorepo. Traditionally if you encounter merge issues the solution is to split your project over more repos
    – Ewan
    Commented Dec 9, 2021 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.