What is captured in a Software Quality Assurance Plan if you are using test driven development / agile methods?

Question

I need to write a Software Quality Assurance Plan (SQAP). I am familiar with the waterfall structure and I have seen SQAP samples based on waterfall model. I also know how TDD works, but I could not find any SQAP that is based on TDD/agile. I assume a SQAP for a TDD is more minimal, but still can anyone please clarify what are the must-have on the document?

Answer 1

TDD is usually practiced along with agile methodologies, which usually does not have all the "Plan" documents associated with the heavier methodologies. That said, if your customer requires the plans, you have to provide them.

It's important to realize that TDD is not sufficient by itself to ensure quality. However, it provides a good foundation for a well tested system. In your SQAP, you'll want to communicate the means by which you intend to show quality which should include sections on at least the following:

• Unit testing: method to test isolated units of code
• Integration testing: method to test code interactions
• Acceptance testing: method to test correctness based on the specification

A plan defines how you use the different types of testing, and when you define the contents of the tests. It's important that the only part of the picture that is more formal is the acceptance testing.

For that, you may want to look at "BDD" (Behavior Driven Development) or Specification Testing. There are some tools out there that let you write your specification tests using language your customer can understand, with hooks created by developers to ensure the test can be automated. They are defined using a construct of given a certain state, when something happens, then the application responds. BDD tools can also generate your specification documentation, along with the traceable test matrix as to what passed or failed.

While the order of whether a test is written before or after the code doesn't really matter, the correctness of that code does. Using a good tooling structure will help you only need to write your SQAP once, with the actual tests and verifications done by your tool.

Answer 2

When I think of a Software Quality Assurance Plan, I think of a document that answers questions such as:

• On the project, what are the roles and responsibilities of individuals with respect to software quality assurance?
• What deliverables will the software quality organization produce and when will these deliverables be made available in different states (draft, baseline, revisions performed)?
• How is the quality of requirements assured? Consider how requirements are captured, how they are reviewed and approved, how they are traced to tests.
• How is the quality of code assured? Are there coding style guides or analysis tools being used?
• How is the quality of product assured? Consider traceability of changes to requirements or defect reports, peer review process. Also consider various levels of testing - unit, integration, system, acceptance. Who owns these tests? How is appropriate coverage determined?
• How is the quality of documents assured? Consider when documents are created, where they are stored, when they will be drafted, baselined, and revised.
• How are defects identified and tracked? Consider both internally detected defects and externally detected defects.
• Every project has a defined process. Will the process be audited to ensure that it is being followed? If so, how often? Who will perform the audits? How are findings addresses?

All of these questions are just as valid in an agile approach as they are in a plan-driven approach to project execution. The only difference may be in the answers. For example. each of these may happen at certain parts of the iteration instead of once at a given milestone date.

In addition, in favor of "individuals and interactions over processes and tools" and "working software over comprehensive documentation", many of these answers may be captured in a lighter weight format. Perhaps a controlled wiki instead of paper documents with formal sign-offs.

One good rule: Document what it is that you do and then do what is documented. If they are out of sync, get them back in sync.

Answer 3

Testing has many parts. Tests written during TDD is only one part. Your plan needs to include all the other parts.

For example, TDD tests typically test just a single small feature, and possibly in just one environment. Do you have a plan for integration tests to make sure all of the features work well together? Does your plan include testing in a staging environment (versus the developers own machine)?

What about a plan for final acceptance testing? Are you going to write automated end-to-end tests? What about exploratory testing? You need to plan for that. What about testing after deployment? Presumably most of your testing is on local dev boxes or on staging servers. Do you have a plan for testing once the software is on production servers? You typically can't run a full suite of tests since you have to preserve the existing production data.

What about testing that encompasses the look and feel? Do you have a plan to do any sort of testing to make sure the colors look they way the were designed to look, and that the images are all correct, non-placeholder images? If you are building a web app, does your testing include all of the different browsers that your users use?

What about when bugs are discovered and fixed? Do you have a plan to account for bug fixes and new features being added?

What about testing metrics? Do you have a plan to track common testing metrics such as code or branch coverage?

... and so on.