6

So we're pretty good at eliciting requirements from our end customers, business division and sales/marketing. We plan projects to add/enhance features to meet those requests and prioritize tasks to meet deadlines in most profitable order.

Over time, the software product has grown, the market has changed and the original customers are less of a priority compared to some new opportunities.

So, the requirements, and therefore specifications, have changed, although all the requirements of all the stakeholders can co-exist (there are no conflicts).

We've also had some churn in the development team with people moving up, across or out and so now there's nobody who was there "at the beginning".

This has led to some customer outages where we removed what looked like unused code paths (the logic was unfamiliar to everyone) only to find the customer that needed that logic happened to run through the application a couple of months later.

We know that we could put in Unit Tests to verify that behaviour does not change accidentally (as in the case above), but this is a large code-base so it is taking time to unit-test everything.

What non-manual techniques/technologies/best-practices are available to us to verify that the software (still) meets the specifications and requirements?

I've heard agile/iterative proponents say something like "the Unit Tests are the specification". Is there any thing we could use to marry the unit tests to the original requirements?

Footnotes

(I saw this other post, but that is less about verification and more about usability testing)

We already do iterative development, releasing every two weeks or less.

The source base is mostly C++ on Unix. Requirements and specifications currently documented in Wiki, Word, ticketing systems and MS Project.

Improve this question
2
  • 1
    I was wondering why this post title differentiates between requirements and specifications - So do you mean that specifications is what was produced at design stage and requirements is what was produced at analysis stage? Or does specifications mean the current running application?
    – NoChance
    Commented Oct 12, 2011 at 10:58
  • 1
    "We already do iterative development, releasing every two weeks or less." Is that a production release? How often do you release a test version to a user, within that two weeks?
    – pdr
    Commented Oct 12, 2011 at 13:18

3 Answers 3

Reset to default
2

If you really want to do this right, you have to accept that there's going to be a lot of manual work. Unit/system/integration/regression testing is vital, but it'll only verify that the software works the way the tests require. Requirements traceability means that each requirement gets an ID of some sort, and that ID is traceable throughout the development process. A requirement ID has to be tied to one (or more) features, features have to be tied to a design, and designs are tied to tests. Ultimately, you can look at any part of the system and trace it back or forward. If a test fails, you can see what requirement isn't being met. Look at some code and see why it is required and what parts of the system use it. If a requirement gets dropped, you can identify what code and tests can be removed. Unfortunately, there's no good tool support for this (we had a bunch of Perl scripts that wired FrameMaker, a custom change management system and a design tool together).

Improve this answer
1

You have two things here: Verification and Validation. The former deals with making sure that the code matches the specifications. The latter deals with making sure that the code matches the customer expectations. Behaviour driven development and Agile methodologies aim to reduce the disparity between verification and validation.

Validation, can be done in terms of scenarios that testers can run through. Those can be automated or not. Beta testers are a good second line since customers can get to play with the new version. Finally, user acceptance tests can be done and is recommended to iron out the last few bugs. A quick iteration time releases during this cycle can be beneficial.

Improve this answer
0

This has led to some customer outages where we removed what looked like unused code paths (the logic was unfamiliar to everyone) only to find the customer that needed that logic happened to run through the application a couple of months later.

That is a strange one. Unless you can prove the code can never be run (ie it is orphaned) why would you do this?

Unless you have a documented list of all requirements, you cannot PROVE the software definitely meets anything.

Improve this answer
1
  • 1
    If there is a list of requirements you should be able to verify that the specifications meets those requirements. Validation of the specification is a whole another matter.
    – Ramhound
    Commented Oct 12, 2011 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.