4

When I write an unit test I usually provide a context (plain object or mocked/stubbed object) that I setup in some ways and then I can run assert statement on the context:

note: code is in pseudo-code; groovy like syntax:

test myTest() {
  def o = getTestContext();
  o.string = "testme"
  o.number = "2"
  assert o.mult() == "testme testme" 
}

But how to organize the test when you need to test a complex boolean expression that takes many parameters ?

EDIT: I have replaced the one line expression with something more readable to avoid confusion.

//this is not a real class, this is an example. Naming is bad, for conciseness sake
//the expression is coming from randomness realm, so it is probably refactorable and simplifiable, but complex real world expression still exists.
enum Type {X,Y,Z}
class C {
  boolean a,b,c,d;
  Type t;


  boolean isEnabled(boolean anotherFlag) {

    def condition1 = (a || b)
    def condition2 = (c && d)
    def goodType1 = t == X || t == Y
    def goodType2 = anotherFlag && t == Z || t == Z && !condition1 

    return (  condition1  || condition2 ) && (goodType1 || goodType2) 
  }
}

All tests for this kind of methods I've read so far are very verbose, not complete and hard to understand.

And it is quite a shame that such a small line of code, even if it is 'complex', generates awful tests.

I've tried to break the boolean expression into smaller sub methods, but sometime it is not so convenient and the permutation count is still high. I also usually break the expression into intermediate variables, but this is not helping in unit test world...

How should I test something like this to have test code matching the briefness of the tested code and the completeness that must assert that my code works as expected ?

Edit: about the retained solution.

Refactoring is indeed a way to go, but I really don't want to 'unroll' all combinations manually in my tests: it is verbose, ugly and hard to understand.

However I'll keep this answer as a preliminary mandatory step: break the expression into smaller pieces before anything else.

Once the refactoring is done and the test still result in combinatory tests, then I'll use the TruthTable solution proposed. I'll just generate the combinations and not declare everything.

I found this interesting, but outdated, article about test combination in Groovy.
The tool they spoke about is out of search engine radar, so it must be dead !
However I'll use the same pattern:

assertThat(permutations, expectations, instanceObject)

Where

  • permutation is all possible values to assign to properties in a 'condensed map': (a:[true, false], b:[true, false], ...)
  • expectation is all combinations that return a specific value, all other combination will be checked against a default value.
  • And you think the code you're testing is easy to understand? Unit tests are often longer than the code they're testing; what makes the unit tests for this code more "verbose" than others, relative to the complexity of the functionality to be tested? – David K Jun 27 '14 at 13:28
  • a) this is an example; not real case; but often boolean expression are quite complex, even when reduced with intermediate steps: it' a bit like regular expression. b) I'm concerned when the test is more verbose and more complicated than the tested code: first it is a nightmare to maintain, and where are the tests for the tests then ? – Guillaume Jun 27 '14 at 13:55
  • 1
    When you code something concisely, quite often it will be more concise than its unit tests possibly can be. How is that a bad thing? Complicated unit tests are often not good, I agree, but the solution may be to allow them to be even more verbose. And they only need to be "maintained" if they are wrong or if you decide to change what the function should do, in which case, by showing you various outcomes you changed, they help you decide whether you really wanted to change it that way. – David K Jun 27 '14 at 14:08
  • So if you need to test the output of a method that takes 3 boolean parameters, you will test every possible combination ? And what if one of the parameters is a not boolean and the amount of possible combination explodes ? – Guillaume Jun 27 '14 at 14:19
  • 1
    That said, I do like to make lists of "test vectors" like the rows of the truth-table approach in the answer by @jk. It's then a matter of deciding when you have a set of test vectors that covers all the possible cases you should. – David K Jun 27 '14 at 14:42
11

One quite intuitive way to handle this is to code a truth table into your test so you have something like:

//last in tuple is expected result, rest are inputs
test date = new List<Tuple<bool,bool,bool,bool,string>>()
{
   {true,true,true,true,"foo"}
   {true,true,true,false,"bar"}
   etc...
}

i.e. a data driven test. For big table you can move this to a data file which you slurp up. The test itself then simply needs to iterate over the table and plug the inputs in and check the expected output.

This also hints at a possible refactoring of the code. An input of Tuple<bool,bool> is essentially the same as an enum with 4 possible values.

  • I started to consider this option after having some trouble to write tests for heavy boolean expression. I considered the refactoring approach as suggested by MainMa, but then I considered truth tables (and generator for the parameter combinations). I asked the question here because I wanted to have hints from others before going to deep in this solution. – Guillaume Jun 27 '14 at 10:28
  • +1, but the problem remains, he has a way too complicated method in terms of testing. – Silviu Burcea Jun 27 '14 at 11:02
  • @SilviuBurcea sure, I've mentioned one possible refactor, there are others though, e.g. extract method which MainMa covers – jk. Jun 27 '14 at 11:14
  • You can do both. Make functions that don't try to do too much by themselves, then write lists of tuples that cover all the cases you need to cover for each function. For very complex input I've resorted to the "data file" technique but regretted it because the file I/O made the unit tests run slowly. – David K Jun 27 '14 at 14:45
  • @DavidK yes for unit test it might be too slow with a file, this is more target to integration tests – jk. Jun 27 '14 at 15:04
10

Your mistake is to assume that the code is short. The fact that it's one-liner doesn't mean it's simple to test, to debug and to maintain. If I had to maintain this code, I would have WTFed quite a lot about people who like writing cryptic, condensed code (also, I hope your a, b etc. variables are just for an example, and in real life, you use more meaningful names).

Here's what it looks like when refactored, isSomething, isAlso and isSomethingElse should be replaced by meaningful names which document your code:

boolean isSomething() {
    return (a || b) || (c && d)
}

boolean isAlso() {
    return t == X || t == Y
}

boolean isSomethingElse(boolean anotherFlag) {
    return anotherFlag && t == Z && a
}

boolean isEnabled(boolean anotherFlag) {
    if isSomething() && isAlso() {
        return true
    }

    return isSomethingElse(anotherFlag)
}

This code is much simpler than the one-liner, but it also makes it clearer that you'll need more than two-three unit tests to test it.

Number of permutations

Talking about the high number of permutations you would expect:

return a

needs two tests: one where the statement returns true, and one where it returns false.

return a && b

needs either three or four tests, depending on the language. Most languages would be lazy and avoid evaluating b when a is false, knowing that anyway, the result will be false, meaning that you have only to test:

  1. a is true and b is true,
  2. a is false,
  3. a is true and b is false.

Now imagine the number of tests you need for your one-liner. That explains the verbosity.

All tests for this kind of methods I've read so far are very verbose, not complete and hard to understand.

  • I explained the verbosity.
  • As for complete, I imagine you're talking about what I illustrated with three tests instead of four.
  • Finally, it's obvious that your tests are hard to understand, since your one-liner is too complex. Do some refactoring, and test separate methods, eventually using Dependency Injection or moving those separate methods to dedicated classes, if appropriate.
  • Yes, refactoring is the natural way to go. By the way my example was awful on purpose: it's the kind of code I'll found in my 'back to future' project. But sometime the expression is not so simple to decompose, or it belongs in external codebase that is not tested... Another point is that intermediate methods should be private, hence they should no be unit tested; and the permutation issue would stay the same. I would also prefer to have intermediate variables than intermediate methods for small bit like this, especially if the class contains already many methods. – Guillaume Jun 27 '14 at 10:38
  • I disagree about testing private methods. You should test functionality in pieces that are manageable to test. After all, typically practically everything we unit-test is an invisible "intermediate result" from the point of view of the user of the program; so why not test things that are invisible to the "user" of a class, in order to ensure that class does the right things? – David K Jun 27 '14 at 14:02
  • I usually test private methods with introspection, but this is a controversial topic. Have a look here: stackoverflow.com/questions/34571/… – Guillaume Jun 27 '14 at 14:39
  • @DavidK: I wasn't suggesting testing private methods, but rather using techniques like Dependency Injection if the code logic becomes really hairy. – Arseni Mourzenko Jun 27 '14 at 23:46
  • 1
    I'm mainly in the C++ world so things are a bit different there. A friend declaration can give you the access you need for unit testing without adding any code to your release version or exposing your class's internals to other production code. That said, the fact is I very rarely want to unit-test private methods, as there are usually public methods that use them nearly directly and make better unit tests. I'm just reluctant to make it an absolute rule not to do so. – David K Jun 28 '14 at 0:24
0

Let me present another perspective. If your real code block is as complex as your example, it is not really good practice to write it that way.

"Concise" does not always mean "elegant". Sometimes, it just means "hard to debug".

Imagine if that complex condition was somehow failing - it would be downright annoying to figure out the problem. A better way is to have intermediate variables storing the values of the individual condition blocks. Of course, those intermediate variables need proper names to help readability.

Expand that "brief" code block into the corresponding nested if-else series. Every little if and else in that expanded block is a potential test condition. Now you will see that it is not brief after all. It is complex logic that must be treated with due respect.

With regards to your question, it would help to separate out each of these test conditions into its own test. They can share common setup, etc. using functions. At the unit test level, simpler individual tests are better because they help you pin-point the problem to the exact failure condition.

Organize each test as per the 3A pattern. This makes the tests readable and also independent of each other. Here's an example:

void testSummation()
{
    // ARRANGE
    Calculator testObject = new Calculator();

    // ACT
    int sum = testObject.add(1, 2);

    // ASSERT
    Assert.equals(3, sum);
}


Arrange:
Set up the environment for the condition to be tested. This might include setting up some initialization parameters, mocks, etc.

Act:
Run an action on the object. A good practice is to make sure that this action is an independent operation as seen from the API of the object.

Assert:
Check whether the action had the intended effect. Validate the output variables, object state, etc. to make sure that the expected behavior was met.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.