How to properly document functionality in an agile project?

Question

So recently, we've just finished the first phase of our project.

We used agile with fortnightly sprints. And whilst the application turned out well, we're now turning our eyes on some of the maintenance tasks.

One maintenance task is that all of our documentation appears in the form of specs. These specs describe 1 or more stories and generally are a body of work which a few devs could knock over in a week.

For development, that works really well - every two weeks, the devs get handed a spec and it's a nice discrete chunk of work that they can just do.

From a documentation point of view, this has become a mess. The problem with writing specs that are focused on delivering just-in-time requirements to developers is we haven't placed much emphasis on the big picture.

Specs come from all different angles - it could be describing a standard function, it could describing parts of a workflow, it could be describing a particular screen...

And now, we have business rules about our application scattered across 120 documents. Looking for any document for a particular business rule or function in particular is quite hard because you don't know which document has this information, and making a change request is equally hard because once again, we are unsure about which spec to make the change.

So we have maybe a couple of weeks of lull before it's back to specing out functionality for the next phase but in this time, I'd like to re-visit our processes. I think the way we have worked so far in terms of delivering fortnightly specs works well.

But we also need a way to manage our documentation so that our business rules for a given function / workflow are easy to locate / change.

I have two ideas.

One is we compile all of our specs into a series of master specs broken by a few broad functional areas. The specs describe the sprint, the master spec describe the system. The only problem I can see is 1) Our existing 120 specs are not all neatly defined into broad functional areas. Some will require breaking up, merging etc. which will take a lot of time. 2) We'll be writing specs and updating master specs in each new sprint. Seems like double the work, and then do the devs look at the spec or the master spec?

My other suggestion is to concede that our documentation is too big of a mess, and manage that mess going forward. So we go through each spec, assign like keywords to it, and then when we want to search for a function, we search for that keyword. Problems I can see 1) Still the problem of business rules scattered everywhere, keywords just make it easier to find it.

anyway, if anyone has any decent ideas or any experience to share about how best to manage documentation, would really appreciate it.

Answer 1

This is one of the major problems with "big-A-Agile" development: it's primarily focused on development. If system documentation matters to your users (and it usually does), your technical writers and their work-product need to be part of the sprint. In the "small-a-agile" community, we've come to understand that there is more to product than just code. We've integrate testers and testing into sprints, including making sure the testers know enough about what is being coded that they can test inside the sprint. We need to do the same with the technical writers, although I don't know anyone who's doing it yet.

Answer 2

The spec should be your unit tests and knowledge of developers. There should be no "document", that describes what software should do. You probably mistaked user story cards, that many agile developers use for some kind of "spec". But those are short-lived and only used for comunication before this user story gets turned into code. It is thrown away afterwards.

I would question using spec as user stories. I would prefer a system, where business experts have their own document, that describes what the system does, but this document is only theirs. They don't pass it to programmers as "what to develop", they only use it to keep track of what the sytem does. So if they want to change anything in software, they need to create real user story, pass that to developers and update their own document after it is finished.

And I would recomend you looking at Domain-driven development/design. I believe it would be good fit for your problem.

Answer 3

Your questions sounds like a rant on something that you thought it would be an agile way of documentation.

Agile documentation doesn't mean creating a huge mess, where it would be difficult to find necessary information, but it means creating structured (only necessary) documentation.

You haven't said how far you are in the project, also how big it is, but paying that part of technical dept would be good on long run. Therefor, my suggestion is to invest some time, and do what you have to do to make it good. That also means adapting some rules how to write specs, where to store them, etc. so in the future you do not run into same problems.

In order to fix the mess :

• form a directory structure, and put right documents in proper directories. You can divide into architecture, high level design, user stories, testing, and whatever is required
• divide user stories (is that what spec is for you?) by functionality in such a way that it is easy to find every piece of information
• make nice document naming scheme. Do not put some generic G562GDVKASDF235.doc type naming, but for example maingui_panel1.doc, maingui_screenorganization.doc, etc. This way you do not need a document that converts document name into what information is needed.
• write some overview document explaining what to find where, naming scheme, etc
• add document templates

Answer 4

Aren't you forgetting about about the non functional requirements?

Do you have an Architecture Diagram, Deployment Process, Database Schema, Disaster Recovery Plan and System Admin manual that are up to date?

Perhaps some of the above mentioned documents are not needed in your context but I come from a maintenance background.

In the maintenance world we look after mature systems and in many cases there are no technical staff members around who understand the non functional requirements.

In such cases, complete and up to date non functional documentation really matters.

Answer 5

Until a few months ago, I worked for a company that either had a technical writer on each team (i.e., sat with the team all the time) or one that participated in team planning, daily meetings, reviews, and retrospectives even if they did not sit with the teams. I worked there 4-1/2 years and they had been doing this for 3 years before I started there which was from the very beginning of their Agile transition.