9

Based on many sources I do not believe the simple definition that aim of testing is to find as many bugs as possible - we test to ensure that it works or that it does not. E.g. followint are goals of testing form ISTQB:

  1. Determine that (software products) satisfy specified requirements ( I think its verificication)

  2. Demonstrate that (software products) are fit for purpose (I think that is validation)

  3. Detect defects

    I would agree that testing is verification, validation and defect detection. Is that correct?

4
  • 1
    The first thing the books on testing say is that "testing is Not the process of showing that the software works correctly. It's the process of finding defects". And than the books bring numerous reasons to define testing like that. So its rather that verification is the process of finding where the software doesn't meet the requirements.
    – superM
    Oct 4, 2012 at 9:27
  • According to definiton, verification ensures that requirements were met. Actually, books define testing as a process of measuring the quality of software. So if you are checking that system is working (positive) with intention to see whether it works, it is not testing because you do not look for bugs? :) On the Wikipedia: Test techniques include, but are not limited to, the process of executing a program or application with the intent of finding software bugs
    – John V
    Oct 4, 2012 at 9:34
  • I think the best way to identify the bounds of the word testing is to think of testing a hypothesis, in that case you're trying to test that there are no fallacies or inaccuracies in the hypothesis, this is not the same as verifying it's usefulness or validating it's applicability, this is merely a case of identifying it's entire behavior scope, regardless of purpose. Oct 4, 2012 at 14:41
  • Have a "nice question" bonus :)
    – Andrew
    Nov 9, 2012 at 12:39

7 Answers 7

1

I think you got it exactly right.

  1. Verification and Validation are different things and are in fact pretty well defined. Although I don't like the document very much the ISO 9000ff is highly relevant for QA and defines Verification as comparing a product with its requirements and Validation as checking if it actually fits the needs of the customer/user and we all know this can differ.

  2. Both can be done through testing. Verification would lead to tests generated form requirements. Validation leads to test done by Tests without direct reference to requirements. I think this is often called explorative testing. Obviously it must be done by people with a real understanding of the real needs of the users, so alpha and beta testing by real users are obvious options.

  3. On a theoretic basis I guess one could argue anything covered by the first two isn't a bug and therefore finding bugs as a separate goal doesn't make sense. But I think there are things that you can't really verify or validate. For example security: How do you validate or verify that a software system is safe against attacks? Instead you try to find vulnerabilities. This search doesn't verify or validate anything if it fails to find problems, but it finds bugs if it succeeds.

3
  • The issue is that many sources mention that verification is only static, while validation dynamic. Its very confused. What would be functional test then? I would say its dynamic verificiation..
    – John V
    Oct 4, 2012 at 9:16
  • 1
    What sources use this definition of verification and validation? On the other hand I don't know any clear and generally agreed on definition of anything ending in -test. So I don't really know what a functional test is for you. Oct 4, 2012 at 9:19
  • Well e.g. ISO 12207 restricts testing as a validation proccess only.
    – John V
    Oct 4, 2012 at 9:25
3

From Wikipedia : "... In other words, validation ensures that the product actually meets the user's needs, and that the specifications were correct in the first place, while verification is ensuring that the product has been built according to the requirements and design specifications. Validation ensures that "you built the right thing". Verification ensures that "you built it right". Validation confirms that the product, as provided, will fulfill its intended use."

You can't test user's needs and check if the specifications were correct by code. So validation is not done by testing.

Verification supposes that your requirements and design are correct, so you can test it by writing code(testing).

2
  • I would not agree - testing is not just testing code, there is also documentation testing etc. BTW, wikipedia also says: Software testing can be stated as the process of validating and verifying that a software program/application/product.. You validate the program by its executioan and investagion whether or not this is what user wanted.
    – John V
    Oct 4, 2012 at 6:44
  • Actually you are right. Testing process also includes Accepting Testing but I talked about Unit, Integration and System testing. If we think about testing process as a whole, verification and validation is done by testing. Oct 4, 2012 at 7:33
1

For the real world, testing is verification and validation of the software that is meets the software's requirements (business/functional/non-functional). The aim of these is to determine if the software is fit for purpose. Any behavior that does not meet the requirements of the application is a defect - the severity of which will need to be weighted up before determining if the software is fit for purpose.

Low severity defects are probably not show-stoppers to passing the software onto a production type use, High severity may require a fix to be produced. In the real world all software has defects, some are coding issues and others are from missing requirements - which may not be tested for because you cannot test a unknown requirements.

1

There are many definitions of verification and validation. Many people even use the V&V tag to group both in a single activity. The aim is to make sure that software makes right things and makes things right. Whether it is to check compliance with requirements or try to find bugs is not essential at this level.

Testing is one out of many techniques to verification and validation, not the other way. Code review is another one, and formal verification, with mathematical proofs yet another one.

Nonetheless, testing should be performed with the aim to find bugs, not with the aim to check compliance with requirements.

The main difference is in the mind of the tester. It is far easier to build a test case showing that sofware works as intended (checking compliance), than to build a test case showing that software fails (finding bugs).

A great tester is passionate about breaking software, not about exercizing it in a safe way.

3
  • thanks, but dont we also test to show that requirements are met? We make sure the software works (meet specs) and then we try to find defects. So its not just about finding bugs. I remember a book saying that the main aim of testing is to measure the quality, not searching for bugs. As for your first point, code review, math proof etc is testing as well and it is called static.
    – John V
    Oct 4, 2012 at 8:23
  • Defects or bugs exist by contrast to the requirements. The nature of the job is identical. It's only a difference in the way of thinking of the tester for improving its efficiency. As for my first point, there are many definitions of all the terms used in software validation (and a first step when joining a team is to get the local dialect in that team), but the majority of people agree that testing is only a dynamic technique. static testing is an oxymoron, or refer to a different technique, not far from review, where the code is executed in the mind of the "tester" and not by a computer.
    – mouviciel
    Oct 4, 2012 at 8:36
  • mouviciel: oxymoron? I dont think so, static testing means checking for possible defects without execution, which is fully possible (requirements issues, design flaws..). It is not the same to verify requirements and check for bugs: you should test that a field can hold int32 value. Thats testing that it works. Then you can try to enter higher values, that is testing for bugs..
    – John V
    Oct 4, 2012 at 8:46
1

Lets see this from a practical point of view. For testing, you need to define test cases. Typically, you define test cases along the specified requirements, and they should cover "happy day" cases as well as "edge cases" - especially the latter are often defined with the intention of breaking the software. When some of your tests fail, they show up bugs/defects. When you have a reasonable amount of test cases for each requirement, and all that tests pass, you may not have fully proven that all requirements are fulfilled, but you have improved the probability for that, thus done some verification.

So for that part of the question, finding bugs and verification may be just two sides of the same process:

  • tests fail: defects found

  • tests pass: verification done (at least, to some degree, if you provide enough and the right tests)

Concerning validation: as @Mert pointed out, validation can be done by acceptance testing, but not by other forms of testing. Thus testing in general causes no validation, only when done as acceptance testing, by some of the potential users.

0

It all depends on your definition of "verification". For example, formal verification is usually not something done by a QA team, but is instead a responsibility of the developers. Almost nobody does formal verification because of the high cost associated with it (knowledge gap and resources needed).

0

Software testing is not same as QA. You got that right. Software testing overall includes many stages (smoke, unit, regression, integration, user acceptance, etc. ) in itself.

Thus, to assure that software works in accordance to the requirements is the man goal of QA (quality assurance specialist - aka used to be simply called testers years ago). However, it is not just testing. QA ensures that proper set of processes to perform quality check of the product in question are in place, or at least taken into design phase of the project.

Thus, ideally you would expect your QA to verify the application against set of requirements and not just try to test it by breaking the software and finding defects.

2
  • QA is NOT just testing. QA deals with quality of development processes..
    – John V
    Oct 4, 2012 at 12:07
  • QA verifies the application against set of requirements.
    – Yusubov
    Oct 4, 2012 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.