Documentation degrading - how to deal with it?

Question

Important: we have no issues whatsoever with source code documentation. This belongs to regular code audit and is kept up to date. Our problem is with developers documentation (or, "external" if you like), little blog-like tips from programmers to programmers which tend to be once written, often left behind.

---

We use wiki-like system to produce programmers documentation - articles wrote by programmers for programmers describing in a bit more details how particular piece of code works. Those wiki-pages usually include:

• motivations behind design decisions for parts of API (for example; we did this ugly thing because this particular 3rd party library wants stuff to be done this way, because this other library ..., because ...)
• explanation of how we deal with particular common tasks (for example; displaying trivial popup, which needs to reference appropriate application styles, register itself in registry component, and implement some interface in order to be automatically "scanned" by other component)
• good practices (subjective as it is, we do actually write this stuff down)
• environment configuration, required tools and its setup

In general, primarily stuff related to writing code that doesn't fit regular code-documentation due to its size and blog post/article-like nature.

The Problem

As far as introducing this system seemed like a good idea few months ago, nowadays I feel like it's causing more problems than it solves. For example:

• people do write articles... but once code changed, wiki update rarely follows
• lot of scratch articles, wrote by somebody in hurry and left like that
• even though article-request usually comes from project lead, it's hardly ever verified for correctness/composition - which sometimes results in poor quality

The usual degradation. Code changed, wiki stays the same. Next time somebody looks for information, what he usually finds is bunch of outdated, low quality stuff - and is wondering what's going on, whether stuff that he found is accurate or (even worse) which parts of it are. And what was supposed to help, ends up doing the opposite.

At the moment it seems people are aware of the problem, project lead included, but apparently nobody seems to be bothered to do anything with it (or has more interesting stuff to do).

My initial thought was to throw it all into the oblivion (after I got bitten by outdated "tips" few times in a row), but I suppose that might be too extreme. Some information is worth noting and good read sometimes, but the problem is still the same: how you deal with its "up-to-dateness"? Have it linked to source code somehow (so when updated version of file is checked into, author of article gets notified that he might need to revise code/article)? Have designated person "watching over" it on daily basics? Do regular cleanups?

Answer 1

It sounds like you are documenting too much trivia in the wiki.

Document blocks of code and methods in the code. Try to make your code self-documenting so you don't have to make a lot of comments. Writing unit tests can help too.

Document design decisions and architecture at higher granularity in the wiki so the wiki doesn't need to change often or take a lot of work to change. If a lot of people on your team already know the architecture and the team is not growing rapidly then there may not be a strong case for documenting them at all, face-to-face is often the best knowledge transfer.

Rewrite or delete out-of-date information immediately, like dead-code the longer it remains the harder it becomes to spot and the more of it accumulates. If you don't have time just delete it and mark the article as needing rework, it slows you down and is stored in version control anyway.

Document procedures by automating them in a script or installation file. Otherwise keep them in the wiki, but every time someone uses a procedure from the wiki, tell them to try to improve the article or automate parts of the process.

Blog articles belong in blogs. If people want to share their personal opinions and advice create a company blog for that. They probably don't want their articles to be modified and no one will modify them anyway, so don't let them clutter up the wiki.

Answer 2

Documentation should be treated as a deliverable and thus subject to the rules of traceability and acceptance, as well as given the appropriate amount of time to be developed.

It is not uncommon to see people "expecting" software documentation to be a given, when it isn't.

Answer 3

Radical but effective. If somebody wrote a new module but did not document it - reopen the task in issue tracker and if it is necessary prevent from shipping all not documented source code. If you allow developers to treat source code documentation as necessary evil you'll end up with fragmentary and outdated scraps of documentation.

In my recent project we tend to at least track all the necessary third party libraries. If somebody introduces new library, but it is not documented - we roll back the solution until the documentation is introduced. Without such a radical approach there would be chaos. For example an unexperienced developer could use a library whose license is in conflict with our software's license.

Answer 4

If something's changing rapidly, it shouldn't be maintained outside of the code.

motivations behind design decisions for parts of API

This is especially important to keep close to the code. As in, in the same source file. That way, it will be a little harder to ignore whenever someone touches the file, and prompt fewer submissions to TheDailyWTF by people who don't know that the external documentation exists.

The usual degradation. Code changed, wiki stays the same.

This is where "executable documentation" -- unit tests -- becomes very useful. If the code changes and the tests don't change along with it, then the build breaks. Of course, writing tests as documentation takes some skill. But so does writing (good) external documentation.

Answer 5

A good way to deal with the problem, is to make it part of the process. If you have code trace up to/reference the relevant pages on the wiki, a developer can easily find out what might need to be updated. In addition, make the reviewers responsible in a code review for ensuring that the wiki is up-to-date (with regards to the update).

Another way to add it as part of the process -- since you're using an agile model, part of the planning process for iterations, could be to update planned changes in the wiki. The wiki then serves as a 'this is how things should work' resource.

Answer 6

If you are using a .net language, look at (Sandcastle) which takes the XML documentation (/// in C#) and converts it into MSDN help format.

The format includes description, comments, and has the ability to include code samples, as well as some other features. You can output into .CHM, .HsX, .MSCH and HTML/ASP.NET formats. The actual project is added to your solution, and built on the build server. We have done this and deploy to a web site every release, and the consumers love it because the documentation is relevant to the release, and constantly updated.

You can also specify what to include in the documentation. Currently we have 2 projects: one for external consumers which includes just the contracts and appropriate items (classes, enums, etc.), and one for internal developers which includes everything in the API, including private and internal flagged items.

This has become the only documentation we use, as motivations and quirks on using the API can be included in the Comments section of the documentation. The process works well where I work.

Answer 7

I would focus on two areas, 1) The code. 2) None-code notes and document.

1) The code.

Try to make it self-documentation. In the past I found that was often advised but rarely explained well, so...

Instead of this sort of pseudo code:

# Routine by mdd on 7/25/2012
# processes cars for sale
a=0
col = Car.all
Collection.loop |a|
 if a.stat = 'fs' then 
   a+= a.value    
   call easo 
  endif
end

Do code that looks more like this:

accumulating_potential_sale_revenue = 0
cars_to_check = Car.all
cars_to_check.loop |one_car|
  if one_car.sell_status == 'for_sale'
    accumulating_potential_sale_revenue+= one_car.sale_price
    call email_about_special_offer(car)
  endif
end

Use source control to track the 'who did what when' info.

2) The non-code

Use a wiki and markdown format to maintain this information. Make it part of the job. Publicize it, post it and blog it. Set up a standard weekly or monthly meeting to review old and new stuff. Make it a mantra that when someone asks about something the answer is given... along with the thought "should this be in the wiki for the next time someone asks?"