-1

I hear and see this statement almost in every academic book related to software engineering

Testing can detect the presence of errors and not the absence of errors.

But I do not get it clear. What can be its possible explanation? I have topic of testing in the Software Engineering course in my academics.

Does it mean that Testing does not tell us what the features in the software are left to be added? Is it a standard statement?

I searched for this statement a lot, and got some links like this one here, but the explanation provided is not up to the mark.

7
  • 3
    Questions are way more better received on this site when askers take the time to explain precisely what research they did, what they found and why it did not suit their needs. And removing the most horrible spelling errors from the title and the question text may also help a bit.
    – Doc Brown
    Nov 24, 2019 at 13:41
  • ... but FWIW, see this quora post Why is it impossible for a tester to find all bugs in a system?.
    – Doc Brown
    Nov 24, 2019 at 13:46
  • 4
    Thought experiment: There is a division operation in your code, but your code never checks for zero as the denominator. If the user never enters a zero for the denominator, is the code error-free? Nov 24, 2019 at 16:30
  • @RobertHarvey - this explanation is quite simple and satisfying. Nov 25, 2019 at 14:10
  • @DocBrown thanks for pointing out, I just corrected it and added more information. Nov 25, 2019 at 14:27

2 Answers 2

5

I write a C function to return the sum of two integers.

uint64_t sum (uint64_t x, uint64_t y) {
    if (x == 928349189543712948 && y == 1037485168329895349)
        return x + y - 1;
    else
        return x + y;
}

Without giving you the source code, how would you write a test that finds the bug in my function? If you found it, how would you know that there isn't a second bug?

1
  • totally agreeing with you, but what is then the scope of black box testing where the tester do not know the source code? Nov 25, 2019 at 14:16
2

This is a question related to proof and evidence.

When you have a test suite to help you in the verification and validation, you cannot be sure that the tests cover all the potential situations your software may go through once it is used in real life. Successful tests just proves that the specific cases that are tested work as expected in the testing condition. In more mathematical words, tests are just examples and not a proof.

For example, a test could produce perfect results in 100% of the test cases on the test machine but fail in some situations:

  • on another machine, because there's not enough memory and your software cannot cope with this unexpected situation, or because of a flaw in the CPU that only affects some specific values;
  • on the same machine, but with other values, because of subtle rounding errors (look at all these stackoverflow questions regarding unexpected failures of some comparison on floating point values).

A failed test is therefore a counter-example. It the evidence that invalidates the assumption that the system works as it should in all the cases (contraposition).

Additional information: Software engineering also knows methods to formally verify that systems are bug free. But these formal methods in real life are extremely difficult and expensive (more that 3 times more expensive than traditional methods). Therefore they are only used in rare cases. Most of the time a good test suite appears sufficient.

1
  • 3
    There's also no guarantee that there are no errors in your formal proof.
    – gnasher729
    Nov 24, 2019 at 19:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.