3

I'm a newbie with respect to git, but not to version control in general — I've been using CVS and Subversion for years. I've read both the Progit and O'Reilly (Version Control with Git) books, and I think I've got a good grasp of the general concepts. I'm trying to set up git for an existing project, and I'm running into some issues that I need some guidance on.

Background: The project involves an embedded SoC chip running Linux. A third party did the initial development of the application, and they set up four separate git repositories to hold the code:

  • One for the chip vendor's SDK
  • One for the Linux kernel
  • One for a custom API layer created by the third party
  • One for the actual application code.

In addition to this, there are some additional files at the top level that are not (yet) under version control at all.

The first issue is that I'd like to combine all of this into a single git repository. Ultimately, a bare clone of this new top-level repository will be hosted on our own cloud service. I've looked at a lot of material on the net (starting from here) regarding submodule vs. subtree, but I'm still not sure which would be the best fit for my situation. We want to be able to continue to track any changes in the four repositories that may occur on the third party's server, but we need to maintain our own history of local changes as well. Any suggestions?

The second issue is that the local developers have not been doing version control at all, except to take snapshots of the entire project from time to time. To further muddy the waters, the order in which the various snapshots were taken is not clearly documented. Keep in mind that each of these snapshots is of a tree of more than 200K files and more than 6GB of data. However, I'm sure that 99.99% of the files are identical across snapshots. What is the best way to find the differences among these snapshots and eventually figure out the correct order in which to put them?

My current thought is to take the one that I think is the oldest, and commit that into git first. Then create a separate branch for each of the others. The git diff tools should then allow me to efficiently find the differences and figure out what order they should be merged into the master branch. Thoughts?

Improve this question
6
  • Why do you want it in a single repository? Technical/political reasons?
    – Thorbjørn Ravn Andersen
    Commented Apr 8, 2019 at 17:36
  • @ThorbjørnRavnAndersen: From my client's point of view (the OEM of the equipment this software goes into), it's just a single project, and that's how they want to manage it. They don't care or need to know that pieces of it came from different places. They just want to be able to check it out and rebuild it when necessary.
    – Dave Tweed
    Commented Apr 9, 2019 at 12:12
  • Is the git history important or do you just want to use git from now on?
    – Thorbjørn Ravn Andersen
    Commented Apr 9, 2019 at 16:10
  • @ThorbjørnRavnAndersen: The history of the individual subprojects is not important at all -- that can be seen by looking at the original repositories. I'd like to capture the history of the overall project since they were brought together, but that isn't very high on the list of priorities, either.
    – Dave Tweed
    Commented Apr 9, 2019 at 16:13
  • 1
    You cannot both have tracking of the upstream repositories and a single repository holding the files from all four repositories. You can use submodules to sort of glue all four repositories together but it will most likely not work like you want it to. Would it be an option to have the third party migrate to your new layout?
    – Thorbjørn Ravn Andersen
    Commented Apr 9, 2019 at 19:09

1 Answer 1

Reset to default
2

First of all, git can be used to compare the various dumps you have.

  • Create a new git repository. You most likely want this to be hosted somewhere, but you can work just locally on disk.
  • Add a .gitattributes file listing the binary file types you have (this is especially important using Windows where the CR-LF handling may break files), and do the initial commit. This is now your master branch.

For each of your dumps do the following (replace dumpname with a suitable, short identifier for that particular dump - a date tag may be fine):

  1. git checkout master - you should now have only the .git folder, and the .gitattributes file in your local repository.
  2. git checkout -b dumpname
  3. Unpack the dump in the local repository.
  4. git add .
  5. git commit - verify all files are included, and add a suitable one line commit message.
  6. If hosted, git push -u origin (or your work will only be available locally)

(identical files between branches will only be stored once)

When done you have a branch for each dump. You can now use git diff dumpname1 dumpname2 to compare two dumps (or any other git tool). This can identify files that have been moved (even if not 100% identical)

This will allow you to establish a timeline and an overview of the differences. When you have a timeline you can redo this on a single branch in the order you have found ending up with the latest.

I would then suggest backporting the latest local changes into your four clones of the upstream, and create a build environment from those giving the same result as you have currently.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.