Git: Best practices for using updates from one branch in another

Question

My team is working on building a bunch of automated tests for our project. Each automated test targets a "customization" our client ordered to modify an existing website.

The current flow is:

1. checkout development
2. git pull
3. checkout -b unique-customization-test-branch
4. magic code stuffs
5. git commit && git push
6. create pull request into dev

Each automated test makes use of a FooBar.groovy library that I'm constantly updating, creating various functions to streamline the process of building these tests. Sometimes one of my teammates will need a new function added to FooBar.groovy, e.g. verify no duplicate rows exist in a table, and ask me to add that for them to use in their current branch / automated test.

Of note: there's a no-pushing-to-dev policy in place, so right now what we're doing is:

1. I checkout a new FooBar branch
2. add function to the FooBar.groovy lib
3. commit, push, and PR into dev
4. my teammates run git checkout origin/development -- FooBar.groovy to get the latest changes

But then they have to either

• discard the pulled changes to FooBar.groovy any time they switch branches to avoid conflicts and mucking the history, which is a hassle for the git savvy and nigh impossible without help for the git novices and often leads to FooBar.groovy accidentally getting committed in their test branch or
• commit FooBar.groovy to their branch and just try to ensure that they've pulled the most recent version of it into their branch before creating the PR so as to not overwrite newer code

There has to be a better way. This seems super not-optimal. Any advice?

Answer 1

It sounds like foobar.groovy should be an actual component not a source file that it checked in. In that case, turn it into a component with a separate repository and have each team or branch reference a version of that component that they need.

Then it's just a matter of specifying which component version to use, just like any other dependency. Different branches use different versions. NPM install or similar the version your branch requires.

Another thing to consider if you don't want to go down the component route is those foobar.groovy changes should be in the same commit as the branch that the developer is working on. A commit is a single unit of work and that includes code, tests, and any changes to make all the tests work. Then after the the pull request is approved you can handle the merging of that test script as needed.

Answer 2

I believe you're over-thinking this. Don't worry about the fact that you only need one file updated; just update everything. If you had waited a little longer before creating the branch you would have started with the updated everything anyway. So, what you have described is pretty much the same thing nearly all teams deal with daily, mainly:

1. There exists a branch that came off of development.
2. Some new commits appear on development.
3. I want those already created branches to be updated with the latest development.

The answer boils down to simply updating the branches. There are multiple ways to do this, but the most popular are:

# Update via rebase
git fetch
git switch unique-customization-test-branch
git rebase origin/development
git push --force-with-lease # I assume you want to push here?

# Update via merge
git fetch
git switch unique-customization-test-branch
git merge origin/development
git push # I assume you want to push here?

I usually prefer rebase over merge for my own branches, but in your case it will likely depend on what you are doing with the already pushed test branches. If they are used elsewhere in the system, I'd probably just use merge so you don't have to force push. (With rebase the resulting history is "cleaner" but you have to force push previously pushed branches, whereas with merge you can just do a regular push.) If you're willing to force push though, it opens the door of another option:

git fetch
git switch unique-customization-test-branch
git reset --hard origin/development # your branch looks like you just created it
# Now repeat the rest of your script starting with "magic code stuffs"
# If you do this, your push step would have to change to force push

Note you could consider updating your test runners to do any of the 3 options automatically.

Side Note: you may have noticed that I used remote tracking branches instead of local branches (e.g. origin/development instead of development). This is simply more efficient so you don't have to check out local copies of shared branches. I'd suggest you do this in your initial creation of branches as well, for example your first 3 steps could be:

git fetch
git switch -c unique-customization-test-branch origin/development --no-track

I also use "switch" which is the newer syntax for this usage of "checkout". In this case git switch -c is synonymous with git checkout -b.

BTW, your usage of both meanings of checkout in the question highlights the reason the new Git commands of switch and restore were created to replace the command checkout. The equivalent new usage of your command:

git checkout origin/development -- FooBar.groovy

would be

git restore --source origin/development FooBar.groovy

or more concise:

git restore -s origin/development FooBar.groovy

Answer 3

This is an opportunity to re-evaluate whether keeping FooBar.groovy as a separate library is still desirable. Library code works best when the feature set is pretty stable. Consider moving FooBar.groovy into the current repository if it needs to be updated frequently.

People have the option to add new functions to suit their needs as part of their task. The new functions get submitted with the rest of the test code in their pull request.

If you find that multiple people need a new function, someone can create the function as its own distinct task, and submit a pull request into dev. After completing the PR, anyone can pull the latest from dev to get the new function. Competing changes will result in normal merge conflicts. This is a much simpler approach.

Keeping FooBar.groovy outside the repository means it is a dependency of the project. Now you must manage multiple variations of this dependency, and this is the root cause of your difficulties. Sometimes it just isn't worth making something a shared library.

Even if multiple projects use FooBar.groovy, you must weigh the cost of duplicate effort to add functions versus the overhead required to manage a dependency that is constantly in flux. Copying and pasting this to new projects might be a better choice.

Jon Raynor's answer suggests using git sub modules. While I won't repeat what he said, I will provide a little cautionary info.

Git sub modules allow you to manage dependencies in version control. Some technology stacks do not have a good dependency management solution. Sub modules give you something. Other tech stacks have good dependency management tools. For example, .NET uses NuGet. NPM exists for JavaScript/ECMA Script. PHP has Composer. These tools directly address your problem, however I'm not sure if any such tool exists for Groovy scripts.

Be aware that sub modules (and dependency managers) don't simplify your workflow. A pull request to add a single test could span 2 different repositories (one for the library code and another for the test). This requires two different pull requests. It still means you must merge branches in 2 repositories in order to get that one test in your dev branch.

Your team might be better served by simplifying your workflow, even at the expense of duplicate effort maintaining reusable test code.

Answer 4

The component approach is good, but another approach would be to encourage a rebase workflow.

These steps of your workflow would be the same:

• I checkout a new FooBar branch
• add function to the FooBar.groovy lib
• commit, push, and PR into dev

Then your colleagues:

• switch to dev, stashing if required
• git pull
• switch to their branch
• git rebase dev
• git stash pop if required

(you can replace the switching and pulling with a git fetch, but then you have to be more aware of the difference between your local branch called dev and origin/dev)