0

Challenge:

  • I am implementing a microservice/app/applet.
  • Several features will be delivered
  • As things develop later features rely/depend on earlier Pull-Requests (PR)
  • Developer resoruces are finite, so a PR is not necessarily merged with main in a time-frame that permits me to commence work on the feature/next.
    • For instance, one or two people work a 3 or 4 day week.
    • Changes and comments on a PR will require updates to the PR; further delaying merge-s.
  • My deadline means that I may not practically wait to code feature/next during this time.
  • As well, it is better to track main with git pull and avoid later complications.

I doubt that these points are unusual. I am looking to offer a more concrete example. The example not not really a microservice example ... Just shows how dependencies work for real-world situations. Such as a Service/Feature => (dumb) Farm:

  1. Infrastructure (PR #1)
    • Buildings feature
    • Plant (equipment) feature
    • Crops/Plants feature
    • Livestock feature
  2. Beef herd (PR #2)
    • requires Livestock
    • Cattle
    • Beef feature
  3. Dairy herd (PR #3)
    • requires Livestock
    • requires Cattle
    • Milking
    • Calving
    • etc. ...

An imaginary PR #4 is going to languish while the first three are reviewed, updates and re-reviewed. Just normal code-review life-cycle and I realise that. I actually want that peer critique, etc.

  • I also want to get on with PR #5 while the PR 1..4 are in-process.

Objective:

Most of all, I would like to find ...

  1. A branch model and procedure to support/enable use of artifacts not-yet-merge-ed PR-s in new work?
  2. Avoid as many merge conflicts as possible.
  3. Intuitive and logical.
    • I appreciate there are technical methods to completely revise a Git tree, etc; I'm looking for a mundane process to avoid hacks on the Git DAG, etc.
    • The best answer is something people using it don't need to think about.

Considerations...:

A few things we've tried and found unhelpful, OR alternatives we've not yet had time to look into:

  • Can't just branch from main -> WIP
    • -> feature/01
    • -> feature/02
    • Git/Github appears to bring everything on the WIP branch into each commit, like concatenation.
  • Wondered if there is a plain way to "borrow" un-merged files from a supporting branch;
    • And exclude from a commit
    • Or incorporate in a comit with: as-identical-to.
    • For something global like this it might be better to 'borrow' using an intermediate branch (a branch/tmp) and later 'un-do' that merge(??) step.

While those thoughts may be impractical. I believe it is a good thing to seek an answer. Looking forward to your thoughts.

1
  • So you need the ability to use artefacts based on other branches? If so it seems to be more an issue of your CI process needing to build and publish versioned artefacts from those branches rather than anything to do with git/branching? Sep 29, 2021 at 7:59

2 Answers 2

3

No Git branching model will let you cut through insufficient project management. You can't solve social problems through technical means.

A secondary problem is that Git's data model does not fit your way of thinking about development. In Git, each commit is a snapshot of the project state, along with the history that led to that state. A commit is not a diff – a diff can only be made between different states/snapshots. In particular, consider the following history where we have a main branch from which a feature A is branched. In turn, feature B is branched from feature A:

main  x---x---M
               \
feature A       x---x---A
                         \
feature B                 x---B

When you want to merge A into main, and B into main, the diff will contain all the changes. In particular, the diff of feature B will include feature A since A wasn't merged yet. This typically makes review difficult. What typically happens is that A is merged first, and then B can be rebased onto the new state of the main branch:

main  x---x---x-----------M'
               \         / \
feature A       x---x---x   \
                             \
feature B                     x---B'

What can you do to avoid this problem?

  • reduce cycle time
  • make your pipeline wider
  • make rebasing easier

The core issue here is that you have multiple pull requests and changes that depend on each other. There are different strategies to deal with this, and most apply on an organizational or project management level, not on a technical level. In practice, you will likely want to use a combination of these strategies.

Reducing your cycle time. Having pending pull requests is not a problem. But it is a problem if this impedes further work. You might therefore want to look for ways to reduce the time needed to merge a PR. That your team is small and low capacity does limit how quickly you can merge, but not as much as you might think.

In order to make it easy to merge PRs quickly, consider that the scarce resource is time and attention from other people on the team. You therefore want to look for ways that minimize utilization of this resource.

  • Have design discussions before implementing the code. Some code reviews are unnecessarily stretched out by debating how something was implemented. This is wasteful, so reaching agreement about the general architecture before writing the code can speed things up. If the appropriate design is unknown, consider a throw-away prototype or spike to evaluate a potential design.

  • Make PRs reasonably small so that they are easy to review. Use comments and commit messages to guide the reviewers.

  • Use automated review tools that minimize the need for human reviews. Linters, auto-formatting, automated test suites, and code coverage checks can help you detect common problems before asking someone else to review the code.

  • Does this even have to be reviewed? If the team members can trust each other to make good decisions, stringent review is unnecessary. The pull request workflow is most useful in open source projects where contributions might not come from a trusted core team. Within a company or trusted team, the value of PRs is more about knowledge-sharing.

    It might be acceptable to let developers merge simple changes without review, especially if this is an easily reversible decision – not a “one-way door” decision that commits you to a particular design. For example, internal data models can be refactored at any time, but public APIs represent commitments that cannot be taken back. Let developers explicitly request reviews where they need it.

Make your pipeline wider. One of your problems is that you have many features that depend on one another. Stalls in feature A will impede progress on features B, C, D, E, and so on. Try to break such dependencies as far as possible.

Sometimes, it is possible to start by defining a shared data model which then allows features to be implemented separately. At the end, the features are integrated with each other. This does introduce some risk (the data model might turn out to be inadequate, the integration might reveal unexpected problems). But sometimes, these risks are more than outweighed by being able to develop and merge multiple features independently. Yes, this requires some upfront design discussion in the team.

Quite often, a software does not have a single feature set, but multiple areas where development is needed. This should be reflected when planning what work to do next. Sometimes, it might turn out to be faster to work on multiple unrelated features in parallel, so that you have something to do while waiting for reviews for the other features. But this will always be less efficient than concentrating on one area at a time, so that reducing cycle time is probably more valuable.

The Kanban project management methodology has a concept of “work in progress limits”. It doesn't make sense to add more and more pending tasks for the “waiting on review” state. Kanban can help see bottlenecks in your process. Eliminating bottlenecks improves cycle time. But a Kanban-managed project might have tasks across separate “swimlanes” that don't interact with each other. If a WIP limit is reached, it is possible to continue making progress on tasks in other states and tasks from other swimlanes.

Making rebasing easier helps to deal with frequent merges and rebases. To a large part, merge conflicts can be avoided simply by coordinating up front who is working on what, and coordinating changes more closely if it is clear that multiple PRs will touch the same code.

But there are also some ways to use tools to speed up dealing with frequent merging and rebasing. Previously, I said that a commit is a snapshot, not a diff. But when rebasing, Git creates new commits by replaying the changes produced by a commit on top of a new snapshot. By using the git rerere feature (“reuse recorded resolution”), Git can remember how you resolve conflicts. This can be extremely helpful when rebasing the same code multiple times. So when you have a sequence of dependent features A-B-C-D, then:

  • when A is merged, rebase B on the main branch, rebase C on B, rebase D on C.
  • when B is merged, rebase C on the main branch (can reuse conflict resolution), rebase D on C (can reuse conflict resolution
  • when C is merged, rebase D on the main branch (can reuse conflict resolution)

Ultimately, rebasing multiple dependent branches will still take O(n²) effort in total so minimizing such dependencies is important (whether by reducing cycle time or by widening your WIP pipeline). But git-rerere can make this effort more bearable. In principle, you might have tooling to automatically attempt a rebase of dependent branches when a PR is merged.

1
  • Thank you @amon that is a considered answer and not an unfamiliar one. I've been tilling the soil in this disclpline for enough decades to agree (a) there is an organisational aspect to the challenge; also (b) it is not just people there is a resource constraint. I does and will always require people-time to do decent reviews, etc. My question is: best technical approach to the people resource constraint. Accepting your premis, NONE or at least not enough. As I say, "it is a good thing to [strive for]".
    – will
    Oct 2, 2021 at 1:05
2

First, this sort of workflow is a lot easier with merges than rebases. The more non-linear your actual development is, the more conflicts you're going to have trying to make the history look linear. You didn't say if you're wanting a rebase workflow, but it's a frequent assumption.

If you are working on feature b that depends on feature a, you can just branch b off of a instead of from main. Yes, this pulls in the entire history of a, but that's okay because a has to merge first anyway due to the dependency.

That still leaves the problem of all a's changes showing up in b's pull request. Two solutions for that are if you merge a before reviewing b, it will show just the changes for b. There's also a "changes from all commits" dropdown on the pull request that lets a reviewer show only certain commits.

You can also base the pull request on a instead of main, then edit the pull request to base it against main after a is merged.

Another solution to your overall problem is to put more planning into making your features independent, or into moving the dependencies earlier. The idea is to choose your pull requests carefully, so that getting one merged will allow you to work for the longest time possible before needing another one merged.

That's often easier said than done, but for example, instead of doing full feature a, then full feature b that depends on a, then full feature c that depends on b, write stubs or interfaces for all 3 at once, then you can spend as long as you want filling in the implementations.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.