4

When migrating a project from one heavily-used library X to another library Y which is essentially equivalent but doesn't have the same API, it may be necessary to make a lot of changes. In the case I'm working on right now, ≈500 additions and deletions: not impossible, but I'd normally say, that's too many for a single commit! (I have in the past had quite some negative experience with too-big commits, and the arising merge conflicts.)

But with only a subset of the changes done, the project doesn't compile. Furthermore, the changes are essentially mechanical: for every function foo previously used from X, there's either a foo' in Y that does exactly the same thing (despite having a different name) or only a very simple wrapper needs to be written.

Is it sensible to first make all the changes and put them in a single commit, or should I better try to split it up somehow? How is this handled when multiple people are working on the refactoring and you can't really wait till it's all done before interacting via VCS – “single-use commits” that are squashed before merging into the master branch, or something else?

Improve this question
5
  • 1
    I'd say that commits should be small in terms of number of feature-related changes, rather than in number of lines. Is there any specific technical reason why you don't want 500 LoC commits?
    – scriptin
    Sep 11, 2016 at 15:07
  • Well, I've spent too many hours with merge conflicts and utterly derailed rebase strands that were down to large commits. That was in a pretty badly managed project though – perhaps I'm unjustifiedly cautious about LoC as a metric for commit troublesomeness.
    – leftaroundabout
    Sep 11, 2016 at 15:13
  • How about coordinating the changes so that no two people are working on the same file or class at the same time?
    – Robert Harvey
    Sep 11, 2016 at 15:19
  • @RobertHarvey of course, but in this case that's not enough: there are lots of functions in the project which accept or pass types defined in the library X I'm getting rid of. All of them need to change; if only one module adapts the types from Y then the modules that depend on it won't compile until they also use the Y types. (In case you point out this could have been prevented by better encapsulation: sure, but that's little use now. Also, I think one can overdo it with encapsulation and this project actually strikes a reasonably good balance.)
    – leftaroundabout
    Sep 11, 2016 at 15:36
  • Then you're going to need to make a big push to get all of that fixed. While the push is happening, the project isn't going to compile. That's just the way it is.
    – Robert Harvey
    Sep 11, 2016 at 15:37

2 Answers 2

Reset to default
5

Splitting a large commit in those squashed single-use commits won't matter from the point of view of downstream merge conflicts - since they're squashed they'll still appear as a single large commit at merge time, including at conflict resolution.

I'd always use wrappers (instead of just re-using the function names). Each and all wrappers would incorporate a single/central control able to switch the entire product between the before and after refactoring. The switch stays in the before position in the VCS until the refactoring is complete. Developers working on the refactoring can turn it to after in their workspaces to work on their portions of the refactoring and turn it back to before when committing the work.

Adding or removing wrappers or wrapper invocations can be done in small/incremental commits, as needed.

This approach allows both the regular work and the refactoring to happen in the same branch, eliminating the need of going on a separate branch just for the purpose of refactoring, thus eliminates the merge altogether.

The great advantage is that you can combine 2 different CI executions (one for each position of the control switch) and get a very precise measurement of how close to target the refactoring is. That would be practically impossible if using a separate branch since a branch merge would still be unaccounted for (always undeterministic, especially for a large refactoring branch).

The above paragraph assumes a CI system capable to patch a small VCS change (flipping the control switch) at the beginning of its execution. If such CI system is not available a tiny branch - containing just the flipped control switch change - can be used instead. Such branch can be automatically sync'd after every commit in the main branch (should be a trivial sync). Yes, an upcoming branch merge would still be outstanding, but it's a deterministic one - a one-liner change.

When the measured after quality level is acceptable and the decision to accept the refactoring is made the control is switched to after in the VCS system and the cleanup work can begin: removing the wrappers, the old library and finally the control switch itself (which also can be done in small incremental commits).

Improve this answer
3
  • 1
    This actually sounds feasible, but how much overhead do all of those wrappers require? How much technical debt will be incurred? Will there be an instinct to just keep all of those wrappers long after their purpose has been served, adding complexity for no benefit other than to alleviate the nuisance of removing them?
    – Robert Harvey
    Sep 11, 2016 at 18:01
  • @RobertHarvey: The extra overhead (wrappers not required as the API itself remains unchanged) should be small (at least in complexity) - merely simple mechanical changes. The (non-trivial) wrappers for the API changes are needed either way. The corresponding tech dept for the cleanup of the extra wrappers themselves (similarly mostly mechanical) should be a relatively small percentage of the overall tech dept accumulated by the refactoring itself (cleanup of the old libs and mandatory/non-trivial wrappers, present either way). The actual cleanup is a matter of discipline.
    – Dan Cornilescu
    Sep 11, 2016 at 19:26
  • 1
    @RobertHarvey: I did exactly this for a framework (and compiler) switch which took us more than 6 months. Since we did not work work in a separate, uncommitted local branch, we always had our product working that time, we could always test the parts which were already on the new platform, and any bugfixes and new features we had to add during the transition could not lead to merge conflicts. Worked very well for us. But indeed, some of the wrappers are still in place now, some years later (some of them are abstractions which are fine for us, the do not actually disturb the development).
    – Doc Brown
    Sep 11, 2016 at 19:41
1

If the transition is going to take on the order of months, it's better to find a way to architect your software so you can have both libraries in your main branch at the same time and make a gradual transition. Use wrappers or feature flags or a microservice model. This takes longer, but is worth it in terms of stability.

If it's going to take a few weeks or less, it's better to do a normal feature branch. When I work on similar tasks with a colleague, we pair program for the initial branch creation, library replacement, and fixing of one file's unit and integration tests. Then we push the branch and split up the remaining work, making commits after every file's worth of changes.

As far as merge conflicts go, what's important is that merges occur frequently, it doesn't particularly matter which direction. If you merge from master into your feature branch once a day or so, you won't have a big scary merge at the end. It's still the same overall number of merge conflicts. That's unavoidable. It's just spread out throughout the few weeks you're working on your changes, and happening closer to when you changed that actual file.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.