Keeping documentation / manuals up-to-date in a rapidly changing project

Question

Background

I work on a relatively large project (~570 KLOC) in a relatively small team (formerly 5 developers, now 2). Large amounts of code can appear and then vanish in months; for example, a few years ago an investment bank launched a new stock exchange, then shut it down 7 months later—thousands of LOC were written to support that exchange's interface, then deleted.

To help operations and support teams as well as new developers, we have an internal wiki (a modified DokuWiki, if it matters) with things like retrospective specs (we're writing specs as we go, unfortunately) and what various settings, flags, and status codes mean and do.

Problem

Due to the fast pace of development, however, much of our wiki content becomes obsolete in months, if not weeks. And it's not as simple as updating a single wiki section about whatever component is being changed; with numerous interlocking components, changing one thing, like a setting in the algorithmic trading component, might require looking through a hundred references to this setting and/or component in the wiki, many whose sections may need updating.

The main issue, I believe, is a motivational one. Developers know that whatever they write has a good chance of becoming obsolete soon, so it's rarely worth it (nor is it practical) to go searching throuh the wiki to update all relevant pieces, especially when 2 or 3 days later there might be a request to change, say, the format of a setting, or even its functionality. This leads to distrust of the wiki—even amongst ourselves, we're never sure whether what we're reading indeed describes the latest functionality. And that, I feel, ultimately feeds a vicious cycle akin to the broken windows theory: the perception of disorder demotivates any action to bring back order.

Idea

My idea was to migrate the wiki to something more like Stack Exchange—for example customizing Question2Answer. For two reasons:

1. I hope that putting all information in Q&A form encourages documentation to be written with the thought in mind, "What question am I trying to answer? Who am I trying to help?" This is unrelated to this post, though.

2. More importantly, my idea is to use tags—a tag for each component, each setting, each business term, etc.—so that it's easier to find posts that need updating, rather than going through every instance of a keyword—and keyword searches are context-less already, on their own.

Question

Before I go through with this idea, though, I thought I might benefit from advice if anyone else has worked in such a context. I'm open to all ideas and alternatives!

How do you keep highly cross-referenced documentation up-to-date in a rapidly changing project?

Answer 1

1. Whenever possible, reduce or eliminate the need for documentation by writing clearly understandable code instead of writing documentation to explain obtuse code.

2. Keep the documentation as close as possible to the actual code it describes. In most cases, that means putting the documentation in the code itself. Some development platforms (e.g. Visual Studio) have external tools available that can be used to convert code comments to other readable forms, such as help files and web pages.

3. Limit the documentation to integration concerns and overall architecture. Let the actual code speak for itself.

Answer 2

Robert Harvey's answer is already a good one, but I like to add a remark about your current plan. The same way your code changes from release to release, the same way your documentation has to change. But if your docs are in a Wiki or a "Stack Exchange"-like system, you cannot integrate this well into your source code control system (SCCS). So I strongly suggest not to separate the docs from the rest of the source tree - the docs should be versioned and tagged together with the corresponding code version. A tool like doxygen can be used to write your docs (including those for end users) in text form (suitable for any SCCS) and generate nicely formatted documents.

Answer 3

Intent is an experimental product attempting to address precisely this issue, currently in an Eclipse/Java envoronment. The goal seems to be to have strongly typed, or perhaps unit tested, documentation. Any part of the documentation becoming inconsistent with the corresponding source code will get flagged up as an error, just like code that calls a function that no longer exists.

This addresses the heart of the problem with documentation; being unable to trust it because it might be older, or perhaps newer, than the system. And unlike simple document generation (javadoc/doxygen), the documentation is structured around readability, not the source code.

Promising as it looks, I'd be interested if anyone has any real-world experience with it...

Answer 4

By not maintaining the documentation, your source code becomes a mess, and over time, the cost of changing it escalates till its too expensive to make minor changes and maintain. At this point you have turned the current value of your code (500K SLOC is worth how much at the moment) into something worth nothing.

Say you code is worth $5Million now, and over 10 years you let if degrade to have no value - that is $500k / year lost. Spending maybe $100K to maintain its value is cheap.

Its really nothing more than simple business economics. Spend some money now on preventative maintenance to reduce costs and loss of value in the future. No difference than painting a steel bridge, only to start back at the beginning as soon as you have finished.