Version control for modification of a part of another project used in my project

Question

There is somebody else's project (A), a part of which (foo.h) I want to use in my project (B) with some modifications. The modifications make sense only in the context of B, so I can copy foo.h into B and track its changes using B's version control. But in the future, I may want to merge new changes made to foo.h in A into B's foo.h. Is such a setup possible in git? If not, is there another tool to manage this?

One hack that I can think of is to make a fork of A, remove all files except foo.h, modify foo.h, and include this repository as a sub-module in B. But I don't want to do this as foo.h doesn't make any sense independently, more so with the modifications that I am going to make.

Answer 1

This is a Difficult Problem™, especially so when using Git.

While many people think about version control in terms of changes or deltas, Git works in terms of snapshots of the current state of the entire project. This makes it difficult to maintain modifications to an individual file that persist across versions – unless we take extra care, updating the upstream version would just overwrite our changes.

So, it might be best to avoid Git here and to either use a fairly manual workflow for upstream updates, or to move part of the merging into the build system. I'll outline a couple of solutions.

The manual 2-way merge. You copy the current state of foo.h into your source tree and directly make your modifications. This is great because it is simple.

When you want to update foo.h to a new upstream version, you start a 2-way diff between your modified version and the new upstream version. The new upstream version might be stored in a temporary file for this.

The problem with this approach is that the diff can be quite confusing since it doesn't distinguish between your modifications and upstream modifications, and it will not point out merge conflicts.

For example, consider that the original header is

#define A 1
#define B 2
#define C 3

And that your modification is

-#define C 3
+#define C 123
+#define D 4

And that the upstream modification is

+// frobnication flags
 #define A 1
-#define B 2
-#define C 3
+#define B (1 << 1)
+#define C (1 << 2)

Then the 2-way diff will show:

+// frobnication flags
 #define A 1
-#define B 2
-#define C 123
-#define D 4
+#define B (1 << 1)
+#define C (1 << 2)

The manual 3-way merge. You copy the current state of foo.h into your source tree, then make a second copy and only modify one of them.

When you want to update foo.h to a new upstream version, you start a 3-way diff:

your modifications ↔ old state ↔ upstream modifications

This can make it easier to spot the different modifications from each source. A GUI tool like Meld or a 3-way diffing tool integrated into your IDE will be quite useful here. You will likely get a screen like the following:

your modifications		old state		upstream modifications
			⬅	// frobnication flags
#define A 1		#define A 1		#define A 1
#define B 2		#define B 2	⬌	#define B (1 << 1)
#define C 123	⬌	#define C 3	⬌	#define C (1 << 2)
#define D 4	⮕

Screenshot using Meld:

Writing patch files. Instead of directly modifying the copied file, you can write a patch file that contains just your modifications. As part of your build process, you would then apply the patch to a current version of the upstream header. If the upstream file changes so that the change described by the patch can no longer be applied, you will get an error.

In a lot of ways, this will not have any technical advantages – you will get exactly the same merge conflicts as before. The advantage is that you now have a textual description of just your changes so that you can put them into version control.

A patch file for the above example might look like:

--- original/foo.h
+++ modified/foo.h
@@ -1,3 +1,4 @@
 #define A 1
 #define B 2
-#define C 3
+#define C 123
+#define D 4

So, a patch is just the output of a diff command.

But manually writing patch files is awkward, so you'd likely create it by copying the header, modifying the header, generating a patch from the diff, and re-generating the patch whenever you update the upstream header.

Git actually has some built-in subcommands for dealing with patches (format-patch, apply, am), but these are somewhat specific to Linux Kernel development workflows where patches are used instead of pull requests.

Minimizing conflicts. Whatever strategy you choose, you will always have to re-apply all of your changes during every update to a new upstream version. To minimize the required effort, you will want to only make very surgical changes, ideally touching as few lines as possible. Typically this means:

• you patch the original file to introduce extension hooks or configuration options,
• but the functionality that makes use of these hooks or options is kept in a separate file.

Do not reformat or refactor the code. If you want to remove multiple lines of code, consider block comments /* ... */ on a separate line or #if 0 ... #endif directives.

Maintaining a patched version is generally a measure of last resort, and other alternatives should be tried first. For example, you can already get a lot of mileage out of some C projects by defining suitable preprocessor macros before including the upstream file. Sometimes, you don't have to change a definition but can add your own. If the upstream project is actively maintained, you might convince them to restructure their code so that it can be used directly without having to patch it.

Answer 2

If all you are doing is merging a single file, version control might be a more complex tool than you need. Instead, consider copying foo.h from the other project into a temp location, then manually merge the file using a merge/diff tool like kdiff3.

Another option to consider is creating a branch with foo.h based on the other project's version of that file. Don't ever update that branch except to update foo.h. After that it is just a matter of regular merges.

cd your_project
git checkout -b foo
cp ../other_project/foo.h .
git add .
git commit -m "Initial version of foo.h"
git checkout feature
git merge foo
# make changes to foo.h and commit
git checkout foo
cp ../other_project/foo.h .
git commit -am "Updated foo.h"
git checkout feature
git merge foo
# resolve conflicts and commit

The only time you would commit to the foo branch is to copy an updated version of foo.h from the other project.

To be honest, I would probably just manually merge the file in kdiff3 or some similar tool. Using version control seems like hitting a thumbtack with a sledge hammer.