Mutation Testing - Exceptions and Infinite Loops

Question

I'm writing a mutation testing system for Clojure and I was wondering what the general consensus is for when a test causes a mutant to throw an exception or enter into an "infinite" loop (my system judges any run of a mutant that's significantly longer than average to have entered an infinite loop) which would not have done so in the original program (i.e. as a direct result of a mutation).

Personally, I would say that in this case the test has killed the mutant, since it has exposed the mutation, however I am no expert and so am not entirely sure.

Answer 1

The goal of mutation testing is to verify that a test suite can detect the kind of errors that the mutation testing system introduces in the code under test. This means that the mutation testing system exercises two pieces of software: the code under test that is subject to mutations and a suite of tests that tries to establish whether the code under test functions correctly.

A good testing framework marks a testcase as failed when an unexpected exception occurs (while a bad framework would just abort the run due to an unhandled exception). This means that if the mutant throws an exception, this should result in a different test result and the exception should not be visible to the mutation testing system.

The non-termination of a testcase is typically not caught by the testing framework, although some may catch it based on mocked functions being called too often.

For the mutation testing system, I would treat non-termination of a test run as a fault that does get detected by the combination of testsuite and human operator.
In my view, if an exception makes it to the mutation testing system, then that should be flagged as a problem in the testsuite, not the code under test.
In both cases, the mutant can be considered as killed.