0

Situation

We are a large team of developers, using SVN for source control on our project. One of our files, FileTracker.cvs, is very large: it contains metadata about almost every file in our repo. The format of FileTracker.cvs is that each row respresents a file, and each column represents some piece of metadata about that file. One of the columns, called "resolved" tracks whether we have processed that file; it can be either true or false.

Problem

Because we use a single file, FileTracker.cvs, to track the status of almost all other files, it becomes a bottlneck for conflicts. That is, many people end up editing it on a regular basis. Because SVN works at the file level, any time two people edit the file, and commit, we get a conflict, even though the edits most likely happen on different lines.

Desired Solution

We would like a solution to automatically merge conflicts in this file, ideally that hooks seamlessly into SVN. Let's say we have a conflict between two versions of the file, Left and Right, and we want to merge into a file called result. Then we just want to implement the following logic:

For each line in Left:

  • If it's not conflicted with right, then line in Result is the same
  • If it is conflicted with right, and the "resolved" column true, accept the line from Left in the Result
  • Else, accept the line from Right in the Result

In the end, we want all resolved=true changes to persist, and we don't really care about the rest of the data as much.

Specs

We are using tortoise SVN and to a lesser extent Eclipse. Our OS is Windows 7.

1 Answer 1

4

First thing I wonder is if you really need a such a file FileTracker.cvs? You did not mention which metadata you store there, but is it really information which has to be maintained manually and cannot be derived automatically from the file itself or from the SVN repo? The biggest issue I see with your current solution is not that all information is kept in one file, but that it contains a lot of redundant information - namely the file names itself. One has to make sure the list of file names in that list matches exactly the list of all files in your project, no typos, no forgotten files, no temporary or backup files which are lying around in your folder structure, but do not belong to the project. I can imagine this might be a little bit error prone.

But let us assume you really need that file, and that it contains information which must be edited manually. So next thing I would ask is if it is really practical to put that information in a huge, central file. If you have a large team of developers, I guess you have also a huge number of files and folders. Maybe it is sufficient to keep the metadata in one file FileTrackerLocal.cvs per folder, containing only the list of the files in that specific folder? That would reduce the number of merge collisions to a much smaller frequency. If you need that central FileTracker.cvs for technical reasons, you can create a simple script which generates it from the local files on demand, which makes in unnecessary to keep that file further in the SVN repo.

Finally, as a last resort, you might embed your own merge tool into your SVN tool chain. This can be a script or program which checks if FileTracker.cvs is the file to be merged. If it is, you can apply your own merge algorithm, if not, delegates the merge request to a standard tool.

4
  • Thanks for your suggestions. I agree that having a single file prone to collisions is bad, but it's not my design choice and not in my control. I figured I would have to write my own tool and hook it in as an external merge, but wondering it there's one out there rather than re-invent the wheel, seeing as what we're looking for is quite simple. Commented Sep 19, 2016 at 8:56
  • @ColmBhandal: my second suggestion does not require that you remove that single file from your design, when you add a simple generator to combine all local files into one (which is I guess much less effort than embedding your own merger). And the fact you won't find a "standard" solution to your problem is caused by the fact you are dealing with a custom file format.
    – Doc Brown
    Commented Sep 19, 2016 at 9:09
  • @ColmBhandal: see also my edit.
    – Doc Brown
    Commented Sep 19, 2016 at 9:14
  • Thanks Doc. Good suggestions- my thoughts exactly :) But unfortunately #1 and #2 won't work, for reasons I neglected to explain in the question for the sake of brevity. FileTracker is part of the core product. It's modified automatically when certain actions are taken in the IDE, by logic that is deeply embedded in a system designed by architects from a land far away in space and time. It's not something my team has the power to change. All I'm looking for here is a means to cope with this unwieldy beast, whose flaws you've correctly pointed out :) Commented Sep 19, 2016 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.