Is there a way to comment a source file using a separate file?

Question

I'm using a large and complex library including R and C++ code, available online via a subversion repository. Trying to figure out how it works I took a bunch of notes in the source code itself, in the form of comments. Years later, the repo has been updated, so now I have to choose among:

1. Updating and starting from scratch (no way..)
2. Updating while merging in order to keep my comments (it promises to be a long and messy merge process, as it's subversion, and the upstream code changed as well in the meantime)

So I was wondering if there is a less painful way to keep personal comments in software while keeping it up to date, possibly using separate files for the comments.

Answer 1

Which comments are you talking about ?

You generally find 4 kinds of comments in the source code:

1. General comments on the "artifact" (e.g.purpose, authorship, license, content of the file, version history, etc..), often put at the beginning of the file;
2. Comments documentation of the implementation structure and API (e.g. each class and function);
3. Useful hints about particular implementation choices or tricks used in the code that would be hard to understand from reading the code.
4. Useless, redundant logorrhea (comments for the purpose of putting a comment, for example: if (c.isEmpty()) // if c is empty

Comments 1 and 2 could easily put in separate documentation files:

• The advantage of having them in the source is a pure proximity convenience: if you modify the structure of a class or the behavior of a function, it's easier to update the documentation directly in the source file. Conversely if a documentation is missing, everybody will notice.
• But inconvenience is the reading or the browsing of documentation scattered across many files. This is the reason why documenting tools such as Doxygen, are so popular for generating HTML or text format documentation.
• Furthermore if you'd maintain an UML model, part of the source code documentation would be redundant wit the model documentation.

Comments 3 cant be but n a separate file. They mostly point out some specifics to know about a very precise group of statements in the code. Their meaning would be lost if you'd extract them, separating them from their code (example: //don't increment anymore or it would go out of bound wouldn't tell you anything if you don't see where it was and what variable was at stake !).

You could get rid of comments 4, especially with clean code (ss R.C.Martin's nice quotes about comment). Unfortunately it's difficult to distinguish them from 3.

How to achieve the transition

It is always a painful process. But you could opt for some tools:

• you could update your comments slightly, in order to generate with doxygen a documentation file and then maintain this file. By the way, if you follow this path and you're reviewing/merging the code, you could also opt for a darwinistic approach: edit the comments you want to keep to have them in doxygen
• you could use an existing comment parser such as this one.
• you could create your own tool, based on this example (to be improved), and implement a brace-counting mechanism that would recognize comments that are outside of a bloc and copy/move them to a documentation file (hint: be careful of namespace blocs)

Answer 2

This seems to be an impossible problem, by definition.

Suppose that you had a set of .txt files, one for each .cpp file, containing your comments - separate ones for each function, of course.

Then when downloading a newer version of the package, you would still be faced with (a) renamed source files and (b) renamed functions within source files, not to mention (c) altered detail within those functions. So you would still have the same merging problem that you have now.

Basically, your comments are dependent on the source code wherever you store them: whether in /*…*/ inside the source code or outside it, in a neatly tagged text file with the same name as the source file. You will always have to re-merge on each update.

That said, build yourself some tools.

1. A filleter, which takes the extra-commented source files you've got, writes the comments to a text file or multiple text files or a database, noting which function and file each comment belongs to, and removing those comments from the original source file. This last is important, because then you can look through the source file yourself to see if any of your comments got left behind, and tune and rewrite the filleter until it really has filleted everything.

2. A stuffer, which takes a source file without extra comments, and the text file(s)/database created by the filleter, and stuffs the comments into the source file in the appropriate place. If it can't find where to put a comment, then it outgribes in one way or another and leaves you to do that bit of the work by hand.

You can easily test the stuffer by filleting the source code you've got and then stuffing it. If what you get back is what you started with, it's all working.

Answer 3

In addition of other answers, you might consider some literate programming approach. Then, your source file(s) -the files you'll type in your editor, e.g. emacs- would be some specialized markup mixing code and documentation fragments, and you'll use one tool to generate the code (it would properly reorder the code fragments to produce a C++ file) and another tool to generate the documentation (including some "structured comments"), perhaps with some cross-referencing.

I once coded a (proprietary) garbage collector using an LP approach. I've got a mixed feeling of it. It is very good for the interface parts (the API), but perhaps too "verbose" for implementation details. YMMV.

Of course, LP is better to use at start of project. You'll spend a lot of time to properly converting some existing code base to LP.

---

^{(I am guessing that you are using a large free software library)}

BTW, to follow a large free software base, you should rely upon its developer community : ask on the mailing list or the wiki relevant architectural questions. Perhaps contribute, at least some documentation notes. In other words, don't expect technical solutions to a non-technical issue (but a social one: a large free software is mostly based upon its community, and its architecture is changing).