1

I'm currently using git to manage a "modpack" for a game, the modpack is made of several smaller mods, each of which is a branch. Some mods may also depend on, potentially multiple, other mods.

These branches will never be merged into master, as they are optional modifications to the base game. When releasing a new version of the modpack, I'd create a new branch modpack-v1.2.3 based on master, then merge all the mods onto it to create the modpack version.

However, my problem comes from managing each mod separately, specifically to do with it's dependencies.

For example, imagine master is the base game, and we have mods a (with commits a1, a2, ...), b and c, where c depends on b. This is my current way to work with these:

master
 \ \ \-> a1 -> a2 -> a3
  \ \-> b1 -> b2 -> b3
   \                 \
    \                 \
     \-----------------b-----> c1 -> c2 -> c3

Where branches a and b build straight off master, while c builds off master, then merges the latest b.

I need to merge b into c instead of rebasing because c could get a new dependency d in the future, and I can't simultaneously be based on b and d.

My problem is that when I update b, in order to update c I need to re-create the whole dependency chain, that is, create a temp branch on main, merge b, then d, then have rebase c on temp.

The rebasing part is, of course, necessary, as I'm updating the base, but the part of re-creating the dependency branch is quickly taking up most of the updating time.

Is there a way to create some sort of "dynamic merge" that, every time b receives a new commit, automatically recreates the dependency chain of c (and any other branches that have b as a dependency) by merging the latest commit of b (+ the other dependencies), and then performs a rebase on the new dependency chain?

2 Answers 2

4

Git is a version control system, meaning: a system for managing the history of your project. It is not suited for managing different editions of a software.

If you want to describe different editions or variants of the game, don't model this with multiple Git branches. Instead, model this within your software and within your build system. Have different folders for your base software and the various mods. Ideally, you would add APIs that plugins can use, or feature toggles to enable/disable different parts of the base software. This is the only approach that is sustainable in the long term.

Git is a very special and opinionated version control system. Branches are just floating labels for commits. Commits don't describe changes/deltas, but they represent a snapshot of the project. Commits are immutably linked to their parent commits, thus forming a kind of blockchain-like datastructure. This makes it possible for multiple untrusted developers to collaborate securely. This makes sense, as Git was created for distributed development in open source projects like the Linux kernel.

A consequence of the blockchain-like immutable datastructure is that you can't just swap out parent commits. You can create an alternate history with the same changes/deltas and then switch your branches to point to the new commits. But there's no way to avoid the merging/rebasing. At most, you can use git rerere to handle repeated merge conflicts quicker.

3
  • I understand the only way to update a branch with dependencies is to create an alternate history and I'm fine with that occurring. The merging / rebasing itself isn't a big issue, usually those don't even have any conflicts, the problem is that if a mod has dependencies a, b and c, every time I update one of those, I need to create a new temporary branch, merge those 3 separately, and then rebase the original branch on the temporary branch. I'm looking for a way to make the process mostly automatic within git (while, of course, dealing with any possible conflicts manually). Commented Aug 23, 2021 at 20:05
  • Also, unfortunately, as many mods are wildly incompatible with each other, dealing with them as pure diffs to an original, outside of the build system makes them much easier to implement. Commented Aug 23, 2021 at 20:07
  • @FilipeRodrigues Git is scriptable, so you can write a script to invoke the correct Git commands in the correct order to guide you through the updates. I still think this is an abuse (or at least a rather “creative” use) of Git. I'd still prefer using a build system to apply the patches for the mods at build time, perhaps literally using the patch tool and storing diff files. The quilt tools simplifies working with larger patchsets and is e.g. used by the Debian packaging system.
    – amon
    Commented Aug 23, 2021 at 20:46
0

Although amon's answer will likely be relevant for most of everyone else attempting this sort of "creative" usage of git, for those that want to use it like this, I ended up using the following scripts to perform all updates automatically:

Warning: This was made for a hobby project with only 1 collaborator, so the code quality is low. Don't use this for anything large or if you're planning on having multiple people work simultaneously, as it will break everything.

The plan is to have a .branch-deps.json file in master that has all of the dependencies of the project, as such:

{
  "a": [],
  "b": [],
  "c": ["b"]
}

Then we have two scripts, one to update a specific branch, and another to update all branches:

update_branch.py:

#!/bin/env python3

# Imports
import subprocess
import json

# Updates a branch `orig_branch` with deps `branch_deps`
def update_branch(orig_branch, branch_deps):
    # Checkout a temp branch on master
    subprocess.check_call(["git", "checkout", "-b", "temp", "master", "--quiet"])

    # Merge all dependencies
    for branch in branch_deps:
        print(f"> Merge`{branch}`")
        if subprocess.call(["git", "merge", f"{branch}", "--no-ff", "-m", f"Merge branch `{branch}`", "--quiet"]) != 0:
            # If we fail to merge, try to `continue` the merge after the user fixes conflicts
            while True:
                input("Unable to merge, check possible conflicts...")
                
                print("Continuing merge")
                if subprocess.call(["git", "merge", "--continue"]) == 0:
                    break;

    # Then rebase the orignal branch
    print(f"> Rebase")
    subprocess.check_call(["git", "checkout", f"{orig_branch}", "--quiet"])
    if subprocess.call(["git", "rebase", "temp", "--quiet"]) != 0:
        # If we fail to rebase, try to `continue` the rebase after the user fixes conflicts
        while True:
            input("Unable to rebase, check possible conflicts...")
            
            print("Continuing rebase")
            if subprocess.call(["git", "rebase", "--continue"]) == 0:
                break;

    # Finally delete the temp branch
    subprocess.check_call(["git", "branch", "-d", "temp", "--quiet"])

if __name__ == "__main__":
    # If any changes exist, return Err
    changes = subprocess.check_output(["git", "status", "--porcelain"]);
    if len(changes) != 0:
        print("You must stash or commit changes before updating")
        exit(1)
    
    # Get the current branch name
    orig_branch = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"])
    orig_branch = orig_branch.decode("utf-8").strip()
    if orig_branch == "master":
        print("Cannot update master, checkout another branch")
        exit(1)
    print(f"Updating branch `{orig_branch}`")

    # Then read it's dependencies
    branch_deps = json.load(open(".branch-deps.json"))[orig_branch]
    print(f"Found {len(branch_deps)} dependencies")

    # And update it
    update_branch(orig_branch, branch)

This script will create a new branch temp on master, merge all dependencies, then rebase the original branch on temp and finally delete temp.

This would be used when you just want to work on a single branch, but it's dependencies have been updated in the meantime.

update_all_branches.py

#!/bin/env python3

# Imports
import subprocess
import json
import update_branch

# Checksout and updates `branch`, with all branch dependencies in `branch_deps` and the already updated
# branches in `updated_branches`
def checkout_update_branch(branch, branch_deps, updated_branches):
    # If we're already updated, return
    if branch in updated_branches:
        return
    
    # Update any dependencies first
    for dep_branch in branch_deps[branch]:
        checkout_update_branch(dep_branch, branch_deps, updated_branches)
    
    # Then checkout the branch and update
    print(f"`{branch}`:")
    subprocess.check_call(["git", "checkout", f"{branch}", "--quiet"])
    update_branch.update_branch(branch, branch_deps[branch])
    
    # And set ourselves as updated
    updated_branches.add(branch)

if __name__ == "__main__":
    # If we have any changes, return Err
    changes = subprocess.check_output(["git", "status", "--porcelain"]);
    if len(changes) != 0:
        print("You must stash or commit changes before updating")
        exit(1)
    
    # If we aren't in master, return Err
    cur_branch = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"])
    cur_branch = cur_branch.decode("utf-8").strip()
    if cur_branch != "master":
        print("Updating all branches must be done from master")
        exit(1)

    # Else read all branches
    branch_deps = json.load(open(".branch-deps.json"))
    print(f"Found {len(branch_deps)} branches")

    # All updated branches
    updated_branches = set()

    # Then for each one, update it
    for branch in branch_deps:
        checkout_update_branch(branch, branch_deps, updated_branches)
    
    # Finally checkout master
    subprocess.check_call(["git", "checkout", "master", "--quiet"])

This script will update each branch in .branch-deps.json. It will also be sure to update dependencies first to ensure every branch ends up updated.

This would be used when master receives new updates, or a major branch with a ton of dependencies is updated.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.