4

Background: Our group of 10+ engineers all share a Git Repository. In the past, someone has checked in binaries that have caused the size of the repository to grow from ~10 MB to ~10GB. These unneeded binaries are slowing downloads considerably. I have found a tool/method for removing the binaries.

Plan: My plan is to copy Git Repository to Git Repository Old in BitBucket, then replace Git Repository with the new, improved, smaller new repository.

Question: What are the best practices for keeping the group from pushing code from Old Repository clones into the New Repository?

12
  • 1
    Why can't you just remove them from Git Repository? – user253751 Nov 13 '20 at 15:32
  • 2
    Yes, altering the history of a repo causes those problems, but creating a new repo also causes those exact same problems, so why bother with it? You need to tell everyone what you've done, and they can work around the problems. – user253751 Nov 13 '20 at 17:27
  • 2
    Then the problem is not solvable. Good luck. Remember, creating a new repo creates exactly the same issues as rewriting an existing repo does. – user253751 Nov 13 '20 at 17:40
  • 1
    Then you have a use case. Get the team to agree that a fix would be nice so everyone is in kn it. I would then create a new repository next to the old without all the history and then use that. Make the old read/only for forensic use only. – Thorbjørn Ravn Andersen Nov 16 '20 at 22:55
  • 1
    You might consider Git LFS. Only the binary files present in the current branch will be fetched, so your data transfer problem may be minimized. – Mike Partridge Nov 17 '20 at 21:42
5

Communication is key. And organization as well.
There is no way you can avoid rewriting history and keep history in sync with all cloned repositories. Therefore your whole team has to cooperate on this to avoid unnecessary issues

  1. Make sure the work created by the cleanup is really important enough to justify it. How often are you fully cloning a repository? If this rarely happens and only with a good connection, just treat it as an annoyance.
  2. Make sure everybody understands the problem and its implications
  3. Agree on a freeze window. This should be at a time when nobody is actively working and there should not be pending feature branches (Note that this can be a major issue)
  4. Everybody delete their clone
  5. Alter the repository
  6. Everybody clones the altered repository and restart working

The critical point here is actually freezing development to be able to wipe the old clones. For 10 people this sounds somehow manageable but a single merge-commit with the old state can bring everything back. Afterwards make sure that this does not happen again, e.g. by adding pull requests and automatic checks

1
  • Thank you Manziel. This is exactly the advice I am looking for. This follows my current plan closely, so thank you for assuring me that I am on the right track. (Kudos for starting your list with 0) :-) – JS. Nov 16 '20 at 18:06
2

Unfortunately you are now stuck with the repo because you care.

If you don't want to be stuck with the repo you need to change the culture in your organisation. This is going to take time, and you will face resistance.

To start with:

  1. Implement Pull Requests
    • Only add those people who undertake the training,
    • and have agreed to fix up any mistake that gets into the repo from a PR they approved.
    • Sell this as a tool to increase interteam communication, and increase the chance of detecting a mistake early. If need be dig through the last two years of mistakes and count how many of them were simple one line mistakes.
  2. Add Automated and unskippable pull request checks.
    • ensure one of those checks is to check for binaries.
    • Sell this as reducing the need for expensive engineers to deal with trivial mistakes. Either by not requiring human review on PR's with known trivial mistakes, or by increasing the speed by which those engineers learn of these mistakes.

The engineers will soon realise that certain actions cannot happen, or else they have to redo their own work. No one likes redoing work, easier to just get it right first.

Some engineers will realise for one reason or another that they need/want to be approvers. They can thus willingly shoulder the responsibility along with the powers. Exactly like an admin shoulders responsibilities along with powers.

2
  • Thank you for your response. We have recently added both of these procedures. Unfortunately, these binaries were added before those procedures were in place. – JS. Nov 16 '20 at 18:01
  • 2
    Fair enough stop the world is the only way this is going to work well. Find a weekend you don't mind sacrificing. Recreate the repo, and inform everyone that they will need to cut over to the new repo. Any work not accepted by XYZ hour on friday they will have to resubmit. – Kain0_0 Nov 16 '20 at 20:59
1

Creating a new repo won't help you. It's basically the same thing as asking people to delete their local repos and reclone. You really need to be focusing on preventing changes like that from getting into the repo in the first place, because that's the problem you're back to.

Your fix is going to alter history, so most of the ways the old changes would be able to get back in are also history altering. If those changes can be made in your central repo without someone who understands git approving, you have bigger problems.

1
  • Thank you for your response. I do realize that the new repo will be the same as the old. I am essentially making a copy of the repo as a fallback in case of issues. I have very little experience altering history and even less in recovering from the problems of altering shared history. I am aware there can be problems, I just don't know what they will be or how to reliably recover. Tips are very welcome. – JS. Nov 16 '20 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.