1

When I've written tests for some code and want to make sure that it's actually testing what it's supposed to, I'll mess up or remove the code-under-test, and see if the tests fail. If they don't, I've got a problem.

Is there a name for this technique? It's a real bother having to talk in sentences everytime I want to suggest to someone that they do it.

closed as primarily opinion-based by user40980, Robert Harvey, gnat, jwenting, BЈовић May 9 '14 at 7:26

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 7
    Part of "test driven development" is to write a failing test and then implement the code until the test passes. This isn't exactly what you are doing, since the code is written but you are modifying or deleting it, but hopefully a decent analogy. – jordan May 9 '14 at 2:54
  • 1
    I disagree on the close votes claiming the question to lead to opinionated answers. I believe mutation testing is an objective name that has been used for this technique for ages and everyone I know of knows this child under this name. – Frank May 9 '14 at 5:08
6

You're looking for mutation testing.

The idea of mutation testing is to check that the code can be modified to cause the testing to fail, as you have pointed out. An example is as such:

public double getDiscountedRate(int age, int employmentDurationInMonths) {
    return age < 30 || employmentDurationInMonths < 18 ? 0.2 : 0.05;
}

So if we have this test:

@Test
public void canGetCorrectDiscountedRate() {
    assert getDiscountedRate(29, 24) == 0.2;
}

The assertion will be true. If the formula is modified as such:

public double getDiscountedRate(int age, int employmentDurationInMonths) {
    return age < 30 && employmentDurationInMonths < 18 ? 0.2 : 0.05;
}

Then we will expect the assertion to be false now. This helps when there may be a careless error in the method, such as (post-operator modification):

public double getDiscountedRate(int age, int employmentDurationInMonths) {
    return age < 30 && employmentDurationInMonths < 18 ? 0.2 : 0.2; // where is 0.05?
}

In this, the fact that the assertion did not fail for the modified code will let the developer know that it was wrongly returning the same discount rate even when the condition is false.

The Wikipedia article on mutation testing has also helpfully pointed out 'fault injection' as a related article.

  • Not sure why this is downvoted other than being way too short of an answer. It would be nice if at least the gist of it was repeated here (see FAQ). Nevertheless, I second that mutation testing is the name for this sort of activity - be it automated (linked tool) or manual (OP). – Frank May 9 '14 at 5:05
  • @Frank not only that, but also due to the useless spammy link in the answer. If you click it, you'll find that it explains nothing of what is asked about – gnat May 9 '14 at 5:31
  • @gnat It seems to me that, starting from the second paragraph, the concept of mutation testing is quite clearly explained. – Celos May 9 '14 at 6:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.