How should I test boolean function with many possible permutations

Question

When I write an unit test I usually provide a context (plain object or mocked/stubbed object) that I setup in some ways and then I can run assert statement on the context:

note: code is in pseudo-code; groovy like syntax:

test myTest() {
  def o = getTestContext();
  o.string = "testme"
  o.number = "2"
  assert o.mult() == "testme testme" 
}

But how to organize the test when you need to test a complex boolean expression that takes many parameters ?

EDIT: I have replaced the one line expression with something more readable to avoid confusion.

//this is not a real class, this is an example. Naming is bad, for conciseness sake
//the expression is coming from randomness realm, so it is probably refactorable and simplifiable, but complex real world expression still exists.
enum Type {X,Y,Z}
class C {
  boolean a,b,c,d;
  Type t;


  boolean isEnabled(boolean anotherFlag) {

    def condition1 = (a || b)
    def condition2 = (c && d)
    def goodType1 = t == X || t == Y
    def goodType2 = anotherFlag && t == Z || t == Z && !condition1 

    return (  condition1  || condition2 ) && (goodType1 || goodType2) 
  }
}

All tests for this kind of methods I've read so far are very verbose, not complete and hard to understand.

And it is quite a shame that such a small line of code, even if it is 'complex', generates awful tests.

I've tried to break the boolean expression into smaller sub methods, but sometime it is not so convenient and the permutation count is still high. I also usually break the expression into intermediate variables, but this is not helping in unit test world...

How should I test something like this to have test code matching the briefness of the tested code and the completeness that must assert that my code works as expected ?

Edit: about the retained solution.

Refactoring is indeed a way to go, but I really don't want to 'unroll' all combinations manually in my tests: it is verbose, ugly and hard to understand.

However I'll keep this answer as a preliminary mandatory step: break the expression into smaller pieces before anything else.

Once the refactoring is done and the test still result in combinatory tests, then I'll use the TruthTable solution proposed. I'll just generate the combinations and not declare everything.

I found this interesting, but outdated, article about test combination in Groovy.
The tool they spoke about is out of search engine radar, so it must be dead !
However I'll use the same pattern:

assertThat(permutations, expectations, instanceObject)

Where

• permutation is all possible values to assign to properties in a 'condensed map': (a:[true, false], b:[true, false], ...)
• expectation is all combinations that return a specific value, all other combination will be checked against a default value.

Answer 1

One quite intuitive way to handle this is to code a truth table into your test so you have something like:

//last in tuple is expected result, rest are inputs
test date = new List<Tuple<bool,bool,bool,bool,string>>()
{
   {true,true,true,true,"foo"}
   {true,true,true,false,"bar"}
   etc...
}

i.e. a data driven test. For big table you can move this to a data file which you slurp up. The test itself then simply needs to iterate over the table and plug the inputs in and check the expected output.

This also hints at a possible refactoring of the code. An input of Tuple<bool,bool> is essentially the same as an enum with 4 possible values.

Answer 2

Your mistake is to assume that the code is short. The fact that it's one-liner doesn't mean it's simple to test, to debug and to maintain. If I had to maintain this code, I would have WTFed quite a lot about people who like writing cryptic, condensed code (also, I hope your a, b etc. variables are just for an example, and in real life, you use more meaningful names).

Here's what it looks like when refactored, isSomething, isAlso and isSomethingElse should be replaced by meaningful names which document your code:

boolean isSomething() {
    return (a || b) || (c && d)
}

boolean isAlso() {
    return t == X || t == Y
}

boolean isSomethingElse(boolean anotherFlag) {
    return anotherFlag && t == Z && a
}

boolean isEnabled(boolean anotherFlag) {
    if isSomething() && isAlso() {
        return true
    }

    return isSomethingElse(anotherFlag)
}

This code is much simpler than the one-liner, but it also makes it clearer that you'll need more than two-three unit tests to test it.

Number of permutations

Talking about the high number of permutations you would expect:

return a

needs two tests: one where the statement returns true, and one where it returns false.

return a && b

needs either three or four tests, depending on the language. Most languages would be lazy and avoid evaluating b when a is false, knowing that anyway, the result will be false, meaning that you have only to test:

1. a is true and b is true,
2. a is false,
3. a is true and b is false.

Now imagine the number of tests you need for your one-liner. That explains the verbosity.

All tests for this kind of methods I've read so far are very verbose, not complete and hard to understand.

• I explained the verbosity.
• As for complete, I imagine you're talking about what I illustrated with three tests instead of four.
• Finally, it's obvious that your tests are hard to understand, since your one-liner is too complex. Do some refactoring, and test separate methods, eventually using Dependency Injection or moving those separate methods to dedicated classes, if appropriate.

Answer 3

Let me present another perspective. If your real code block is as complex as your example, it is not really good practice to write it that way.

"Concise" does not always mean "elegant". Sometimes, it just means "hard to debug".

Imagine if that complex condition was somehow failing - it would be downright annoying to figure out the problem. A better way is to have intermediate variables storing the values of the individual condition blocks. Of course, those intermediate variables need proper names to help readability.

Expand that "brief" code block into the corresponding nested if-else series. Every little if and else in that expanded block is a potential test condition. Now you will see that it is not brief after all. It is complex logic that must be treated with due respect.

With regards to your question, it would help to separate out each of these test conditions into its own test. They can share common setup, etc. using functions. At the unit test level, simpler individual tests are better because they help you pin-point the problem to the exact failure condition.

Organize each test as per the 3A pattern. This makes the tests readable and also independent of each other. Here's an example:

void testSummation()
{
    // ARRANGE
    Calculator testObject = new Calculator();

    // ACT
    int sum = testObject.add(1, 2);

    // ASSERT
    Assert.equals(3, sum);
}

Arrange:
Set up the environment for the condition to be tested. This might include setting up some initialization parameters, mocks, etc.

Act:
Run an action on the object. A good practice is to make sure that this action is an independent operation as seen from the API of the object.

Assert:
Check whether the action had the intended effect. Validate the output variables, object state, etc. to make sure that the expected behavior was met.