How do you balance documentation requirements with Agile developments [duplicate]

Question

In our development group there is currently discussions around agile and waterfal methodology. No-one has any practical experience with agile, but we are doing some reading.

The agile manifesto lists 4 values:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

We are an internal development group developing applications for the consumption of other units in our enterprise. A team of 10 developers builds and releases multiple projects simultanously, typically with 1 - maybe 2 (rarely) developer on each project.

It seems to me that from a supportability perspective the organization needs to put some real value on documentation - as without it, there are serious risks with resourcing changes.

With agile favouring interactions, and software deliverables over processes and documentation, how do you balance that with the requirements of supportable systems and maintaining knowledge and understanding of how those systems work?

With a waterfall approach which favours documentation (requirements before design, design specs before construction) it is easy to build a process that meets some of the organizational requirements - how do we do this with an agile approach?

Answer 1

I wouldn't necessarily say that agile eschews documentation so much as it eschews up-front documentation. This is meant to address the problem that what gets documented with such tremendous effort up front (a) ends up being very different from what gets created because business needs change, scope creeps, etc., or (b) causes the project to fail because delivery is contingent upon such huge expectations.

There are certainly cases where documentation is essential. I just started working on a commercial application, for which providing accurate, up-to-date documentation to customers is essential. The team is also agile (scrum).

In our case, we have one person (non-dev) who is responsible for documentation.

This helps, but there are more critical components. One is making documentation part of your "definition of done." You get acceptance criteria from stakeholders for each feature, but append to that the requirement that the feature be documented. The second component is that you bake documentation into your process. It cannot be optional, it cannot happen "whenever." In our case, we demo a given feature for the documentation person after stakeholder review and before it goes to QA/Testing. Every time. If you've said your backlog item is ready for test and you haven't submitted it for documentation, you broke the process -- something like breaking the build.

This fits better with agile because you are only documenting what is actually in the finished product.

I realize we're talking about different kinds of documentation -- you're talking more about specs. The last part still applies -- only document what you actually put into the product. I'll also add to that the tip that you can create your unit tests as "executable specifications," and let your test suite document the product. You don't necessarily need to practice TDD to achieve this, but if your team isn't already unit testing extensively, then that is a whole other ball of wax.

Answer 2

First, be careful committing to agile without experience its the best way to fail. You do not want to migrate to an agile experience without a team that knows what agile is, and how it is supposed to work. I strongly suggest, before you move to agile find at least one black belt, and send over half your team to agile training.

With that being said, agile doesn't say NO documentation. It just states that the process of agile, reduces the need for extensive documentation. Since there is a tighter loop of feedback, end users generally understand what the system is supposed to do. Most of the successful agile projects I have worked on there has been some type of collaboration wiki/tool that really was the live documentation. Developers/Business Users all actively contributed to documenting the system, and who is to say that you can add to your stories some type of system overview.

From my passed experience, the 40 page manuals are never useful, or only useful for the super technical folks. When you are wasting your last 3 weeks writing documentation, you are missing valuable weeks of transition periods. Time is better spent mentoring others, walking them through live code, helping them support, etc.

As for your final point, agile doesn't state to design before requirements, it just says you don't have to do ALL of your requirements before you start design.

Hopefully that answers your question.

Answer 3

Personally, I insist on 1/2 day in each week being a documentation day (typically Friday am). While new classes/methods/properties should be docblocked while developing, that extra half day allows for annotations in the API docs, and comment in the code. Each set of new comments is peer reviewed to ensure that another member of the team can understand that documentation, before code development is allowed to continue.

I've found, particularly when extending methods with new arguments, that these often don't get added to the docblocks, so the API quickly gets out of date... but this 1/2 day tries to enforce keeping it up to date.

I've also been looking at automating processes to identify when the docblock gets out of sync with the method definition.

Then 1 day in every month is devoted to ensuring that non API documentation is maintained... e.g. working examples, etc.

If these "overhead" days are actually planned into the sprint, then documentation ceases to be a major issue.

Answer 4

Let me ask this. Under waterfall how many times did you write a 40 page design document and when the project was "over", what you had matched that document? How much time did you spend on that document? How many people have actually used it? How many 40 page design specs have you ever found really useful?

In agile we do not waste time spending weeks writing design documents when we could be writing code and delivering it. We are not saying your API's should be a mystery, or you should not make clear what the code is supposed to do. On the contrary we put extra effort into the design of the api's. Making sure methods are well named. That the purpose of a class is clear. That the code is clean and obvious.

I typically find that the last thing consumers of my code want is a big fat design doc. What they want are nice clean sets of example code and well designed obvious usages. Because that's what I would want.

UPDATE: I should also point out that most Agile shops tend to document their code with a suite of acceptance tests in something like Fitnesse or Cucumber. This allows the API to be documented with working code which is constantly exercising it under a CI environment. This way your documentation never gets out of date, because if it did, it would be broken.

Answer 5

The Agile Manifesto concludes with the sentence "That is, while there is value in the items on the right, we value the items on the left more". Meaning: in doubt decide for working software over documentation. But as you mentioned, there is need for documentation in your situation, no doubt about it. Agile is often (mis-)understood as "you should reduce effort in items on the right, otherwise you are not agile".

You should always avoid unnecessary documentation. Agreed. But: you should spend some time thinking about which documentation is necessary, and which is not. Don't be fooled by a role-based, narrow perspective as e.g. a coder. The coder may not need a document describing his architecture. But that doesn't mean it is an unnecessary document for the project's lifecycle. Ommitting it may put you in a highly non-agile situation in only a few months time.