6

Context: I am developing a visual studio plugin that generates layer diagrams. I want the tool to be able to produce an intermediate output, which is the data representation of what is being rendered in the diagram. The idea is that this serialized data can then be checked in together with the source code which would allow developers to more quickly see how architecture changes between commits. This would either be done through general diff tools (like git-diff) or a dedicated diff tool. However i want the file format to be optimized for the first, because of their accessibility. The data would be represented as a tree structure of nodes, where each node would have a name and a list of dependencies (other nodes represented by their path + name).

Question: When designing a file format, are there things regarding what data interchange-format or syntax is used and the formatting of this syntax (use of line breaks for example) that would make different versions of the file easier or harder to compare using tools like git-diff. For example, ensuring that the tool sees a "rename" as a "rename" and not an "delete" followed by an unrelated "add".

1
  • What information, other than what is already checked in to VCS goes into these diagrams?
    – marstato
    Commented Apr 27, 2019 at 21:38

1 Answer 1

1

This sounds like a very cool idea. Can you make one for C++ (nudge-nudge)?

I've done something similar to this before as kind of a poor man's, humanly-supervised testing in rushed scenarios where I just outputted files from manually-executed tests outside of CI with the resulting output files submitted to version control. Then I'd inspect the file diffs to look for unanticipated changes (these were in approximation fields where the changes didn't necessarily indicate erroneous output, but should be monitored carefully).

I can only give some general suggestions without knowing the elaborate details, but my basic suggestion to start would be to favor a format that is as flat as possible.

That is, avoid something like XML here. XML can be great, but it's not necessarily the most diff-friendly or the easiest to read. There's ways to get the nested structure an XML file represents in a flat way. For example:

<foo>
   <blah/>
   <bar>
      <baz/>
   </bar>
</foo>
<x>
   <y value="123"/>
</x>

... might be represented in a flat way as:

[foo]
blah

[foo/bar]
baz

[x]
y = 123

... something like this as a crude example. You can think of it kind of like a file system structure where stuff in [...] is a directory and the stuff below are like files within that directory. It'll be easier to control that way and have everything on its own line, easy to spot differences, etc.

Next easy step is just sort the entries in the file ("directories" at the top-level and "files" within a directory). That'll give your file format a lot of stability so that it doesn't just arbitrarily rearrange the contents of the file and totally screw up the diff, tagging way more lines as changed than necessary.

I think you can actually stop at this point. If you want to go further, you can start introducing extra whitespaces to leave room for new entries to be inserted within a "directory", e.g. (kind of like a growable array container implementation which has excess memory capacity to append elements to the end without a linear-time complexity reallocation and transfer of the previous contents), and even leave "tombstone" whitespace lines behind that you can reclaim when new things are inserted. That'll result in the minimal amount of shifting changes to the file in common cases in exchange for big shifts in rare cases, though I think it's really overkill and will come with the burden on your human readers to deal with excess and seemingly-arbitrary whitespaces. It's also far more complex since you'd have to inspect the previous files you made to modify them instead of just being able to overwrite them.

The sorting and flattening of the format is the easiest way to achieve a pretty good stability of a text format IMO for diff purposes. Last but not least, this is probably just common sense and something you already know upfront, but naturally it'd be good to grab various diff tools and test your outputs against them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.