6

Here's the problem and how we currently manage this at work.

We have a buildout recipe that fetch multiple git repositories. Sometimes, it is necessary to patch a module from a repository that we don't own (public repository). At my previous position in a different company, we used to fork all the public repository and push patches in different branches. This work well but in some cases, it's much harder to maintain and sometimes, patches are really specific to a particular client so it becomes hard to understand which branches are relevant and forking 50+ repositories isn't particularly easy to manage if you have to give permission to push to developers. At the same time, we managed patch files that could be applied directly without forking any repository.

At my current job, I decided to limit myself to patch files because it simplify the process. Technically applying a patch and merging a branch is pretty much the same thing.

Patch are stored on a per client repository and applied in the build process. Since with fetch multiple repositories, some patch are to be applied on projectA and some other on projectB ...

Right now, I'm writing every single patch that needs to be applied in the build config file but I was wondering if there was a way to make it less coupled witht the configuration.

Like instead of applying a patch, I'd apply a patch set that would be more close to a merge that can apply multiple "commit". But the patch set should be able to apply patches in multiple directories/repositories. Patch are usually made for the particular repository using git format patch.

6
  • Can't you group patches to patch sets by writing a script containing the necessary git commands? That script then could be called within the build process.
    – Doc Brown
    Commented Feb 20, 2017 at 15:04
  • I could but it would be the same, I'm looking for a solution that would be more "automatic"
    – Loïc Faure-Lacroix
    Commented Feb 20, 2017 at 15:41
  • Well, you asked for a way to separate it from your build configuration, that is exactly what I suggested. And a script is a form of automation, you can run it inside your build, or outside of it, and you can "apply patches in multiple directories/repositories" in one step (your words!) . So if that is not what you want, you should consider to edit your question and explain more precisely what you have in mind
    – Doc Brown
    Commented Feb 21, 2017 at 9:45
  • @DocBrown by automation, I meant not having to write the script at all. In my build file, I'd have something like "apply patches" and nothing else and I'd expect a tool that manage the patches that format them with command tools. I could roll my own but if there is something existing I don't see why not use that instead.
    – Loïc Faure-Lacroix
    Commented Feb 21, 2017 at 11:08
  • Well, if you are only asking for a tool, you better do not ask on this site, since questions for tools or third party resources are typically closed by the community here quickly.
    – Doc Brown
    Commented Feb 21, 2017 at 11:53

2 Answers 2

Reset to default
4

For the similar scenario we used a Quilt tool - patch set management utility: https://wiki.debian.org/UsingQuilt

Basically you have a patches directory in the root of your project which contains several independent patches, managed by the quilt. Patch dir could look like this:

patches/100_asserts.diff
patches/101_terminate_call.diff
patches/102_status_code477.diff
patches/107_parser.diff
patches/110_ssldefault.diff

Patches are applied sequentially (quilt push), can be unapplied (quilt pop), updated (quilt refresh) and so on. It makes sense to separate patches to files according to their logic.

In the situation with more repositories you can create a new wrapping git repo with dependencies as git submodules.

patches/  <-- patches repo01, repo02
repo01/   <-- git submodule
repo02/   <-- git submodule

In this example, the patches directory is managed by git.

In the single repository use-case we used to fork the repository, add the patches dir and maintain the patches in the same repository.

Updating is also doable by: pop-ing all patches by quilt pop -a, git pull, quilt push patch one by one, resolve conflict if needed, usually quilt refresh does the job.

There are also tools that directly integrates with git:

1
  • That sounds exactly like what I had in mind except that our project structure might not allow it to work easily with this. The main project that manages everything is a subfolder of the whole project. Like this: project/git/client and other projects are also loaded in project/git... But thinking about it, I could technically make a symlink in project/git/client to point to the git folder...
    – Loïc Faure-Lacroix
    Commented May 4, 2017 at 11:35
0

From what I've understood in your question, you may be using patches too often when the best solution maybe to use the versioning system directly. Patches were made just for quick fixes.

So, if we are using repositories we don't own, we may use:

  • patches on a public repository version
    • but ask for our patch integration ASAP in order to use the repository directly without patches whenever possible, reducing the patches to be applied.
  • clone/fork the public repository and maintain our own organic evolution (and e.g. use Submodules to integrate in our full project) [this is equivalent to having patches on the public repo] (how to work with submodules)

If "patches are really specific to a particular client" maybe we don't want to patch these public repositories at all (it was built with another philosophy/use-cases/architecture in mind), and it is best to use alternatives as:

  • do the changes in our code that uses these external repositories
  • create a decorator/wrapper of the external repository
  • use an alternative external public repository (or create our own)

I know you've asked for a patch management tool. But this just seem like a versioning system (SVN, git, mercurial)!

But, if you really need to use a patches system, you may create a configuration file on projectA, another on projectB, (etc) that will describe the needed patches to be used. And the patches will exist in an independent repository.

11
  • We use git for code versioning, patches are usually made for fixes that can't be wrapped which would require to duplicate large part of code in our codebase and that isn't really something we want. As I said, forking repositories is less interesting because we have to manage "branches" with fix outside of the client repository which makes it hard to know which branch on github are really relevant.
    – Loïc Faure-Lacroix
    Commented Feb 21, 2017 at 14:34
  • 1
    Why would projectA need a different patch on the same external public repository (a library) than projectB? Can't you just change the code only at each project instead of at the library? The library must have a generic behaviour...
    – Pedro Reis
    Commented Feb 21, 2017 at 14:52
  • Because having a patch of 3 lines makes more sense than copying a method of 300 lines in which we will change only 3 lines. As soon as something change in the remote repository, we won't notice code changed as we override completely the method in other word, we will be missing fixes/improvement in said method. So copying code is a no go most of the time. Again, fork are not interesting because I can't manage easily 50+ forks. Not all of them have patches and it's a pretty long list to cover. Patches on the other hand are centralized in client repositories and can be shared accross clients too
    – Loïc Faure-Lacroix
    Commented Feb 21, 2017 at 17:22
  • 1
    Yes, but I still don't understand why do those 3 lines must be changed just for one of the repositories? (Why isn't that generic code that all the repositories can use, and each time you change that, you are just augmenting features and not breaking interfaces for the other repos to use?) And can't those 3 lines be applied e.g. in a decorator instead of directly at the external repository? If none of these apply, you can use or create another external repository with the pretended characteristics of that specific client-repository. But I'm aware I may not visualizing your situation properly.
    – Pedro Reis
    Commented Feb 21, 2017 at 17:38
  • Because the 50 repositories are actually a collection of module not including the client ones. These are 50+ repositories that I'm not owner and I'd have to fork otherwise. So yes each repository doesn't duplicate the same code... I wouldn't need 50 times the same thing.
    – Loïc Faure-Lacroix
    Commented Feb 21, 2017 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.