# Are short-circuiting paths considered for path coverage?

In a book Doron A. Peled, he states that "

path coverage does not subsume multiple condition coverage because you can execute all the paths without exercising all the conditions.

But I believe this is not right (it implies path is only "visible" branches), because even cyclomatic complexity is increased by one for each logical operator in a decision, stating that a new decision node is created in the program control flow chart, creating immediate paths that needs to be exercised - in short-circuiting languages. Therefore: if(A && B && C) have two paths from the traditional perspective (branches), but precisely there is more than that if the language allow short circuiting. Thus, I believe the statement quoted is not correct. Even for a single decision, they are equally strong. In the context of a whole method, path coverage is obviously stronger as with every decision the total count doubles.

Am I missing something or is my reasoning correct?

So I understand what you are saying to mean that this:

``````if(A && B && C) { ... }
``````

Has this graph:

Which after drawing it out, I think indeed has to be correct. If A,B and C have no side effects e.g. if they are just primitive Boolean values, then I think you can get away with reduce them to the same graph as the non-short cut version:

``````if (A & B & C) { ... }
``````

But if A, B, and C are functions, you really can't ignore the fact that B won't evaluate unless A is true and C won't evaluate if A and B are not true. So in order to get all paths, you can't just test the situation where A is false and the situation where A,B, and C are all true. You need to test where A is true and B is false, A and B are true and C is false as well so short-circuiting adds two paths.

My first thought was that you are right and this is an error in the author's description. But I just realized this point might make sense if we consider non-short circuiting. Consider:

``````if (A | B | C) {...}
``````

In this case we always evaluate all three conditions so there are only two paths. However, there are 8 different combinations of conditions. We can therefore evaluate both paths without checking all the possible condition states. Perhaps that is what the author means here?

There are a couple kinds of code coverage tools. Most tools have statement coverage or branch coverage. However, some do provide condition coverage. Doron's statement holds true for all coverage tools that don't provide condition coverage. Assuming you had a run where A and C were true, then those two parts of the codition will be marked as covered, but B won't be. That said, it doesn't prove you've fully tested the conditions.

In the case of short circuiting, the questions you have to ask are:

• Do my tests have side-effects? In other words does B or C do things that I'm expecting down line?
• Is short circuiting what I want? In many cases, yes.
• Have I tested the semantic meaning?

If your tests have side-effects, that can lead to very hard to find bugs, particularly if down-stream code is expecting B's side affects to be applied even if A is false. It's always best to minimize this as much as possible. I won't say eliminate it since you may be counting on the short circuiting to avoid a costly action if possible.

You haven't semantically tested your condition unless you have tested all designed eventualities. In other words, in this case there may need to be as many as 9 tests to fully evaluate the semantics of that branch--particularly if you have side-effects you are counting on. Of course, if you can prove that the semantics really are that everything must be true to run the code inside the if statement or it doesn't get called at all, it might be semantically correct to only have 2 tests.

With good design you can separate code that manipulates state from code that checks it. That will simplify the semantics greatly for your `if(A && B && C)` to `if(true)` and `if(false)`.

• Sorry, but I believe you are talking about branches, not paths. Apr 27, 2018 at 12:44
• An if statement if(A&&B&&C&&D) has cyclomatic complexity of 5, therefore there are 5 paths to be tested in short-circuiting languages, but only two branches (True and False). Apr 27, 2018 at 12:51
• @user970696, read more carefully. I am talking about both, so it sounds like you have decided on the answer you want to hear. I mentioned that if you are going to test every combination of of your condition, you have to have n^2 tests to be complete. However, if you can prove your semantic meaning of the evaluation is satisfied with fewer combinations, you can use fewer tests. The most basic reduction is going to be true and false. Apr 27, 2018 at 13:07
• Your answer does not contain even the word "path". Reduction to T and F leaves out the side effects done by short-circuiting, which is why CC mandates you to have 5 tests written for if(A&&B&&C&&D), and not just two. Apr 27, 2018 at 13:11

Consider the hypothetical refactoring where we replace

``````function(bool A, bool B, bool C) { ..
if(A && B && C) {..
``````

With

``````function(Func<bool> conditionFn) { ..
if(conditionFn()) {..
``````

It doesn't matter how complex `conditionFn` is, `function` has only two paths.

But! you should obviously not only test both paths, even though this would give you 100% path coverage, but also whether the correct path is followed for given input parameters A, B and C. ie. tests for `conditionFn`

if however we have

``````function(Func<bool> A, Func<bool>B, Func<bool> C) { ..
if(A() && B() && C()) {..
``````

Then, yes we have more branches due to short circuiting possibly preventing the code in these functions from running if for example A returns false.

• I am not sure I follow. Yes, if there is no boolean operator within the decision, then there are truly only two paths. The problem here is specifically with decisions with logical operators in short-circuiting languages, which generates additional paths (and that is why cyclomatic complexity gets increased by 1 for each logical operator). Apr 27, 2018 at 14:29
• by that definition of CC, if the microchip checks each bit of A in turn the CC is increased by the length of the datatype. In the code as written, there are only two paths
– Ewan
Apr 27, 2018 at 14:53
• Yes, if(A) has two paths. If(A && B) has 3 etc. The question is, whether this implies multiple-condition coverage. I think so, because those paths are executed in the combinations of conditions. Apr 27, 2018 at 14:56
• I think you are confusing `condition` and `branch` coverage. there are only two code paths, you can get 100% branch coverage with 2 tests. for 100% condition coverage you need 8
– Ewan
Apr 27, 2018 at 15:03
• Well, McCabe defined it quite well. I see your point, although in short circuiting it does make sense, because you might have unwanted side effects if only part of the expression is evaluated, which is this additional path. Apr 27, 2018 at 15:13

The statement you quoted is true, but not directly helpful due to the prevalence of short-circuiting operators in modern languages. It is not correct to interpret `&&` or `||` in C-like languages as logical operators. Instead, they are control-flow operators and a sequence point. Other languages make this clearer (e.g. the logical `and` vs. the control-flow `andalso` in Standard ML).

For a program using only short-circuit operators, it is possible to interpret multiple condition coverage as equal to branch coverage and equal to simple condition coverage. Then, path coverage subsumes multiple condition coverage. This interpretation is essentially correct, but not very useful.

Multiple condition coverage is not fundamentally about the control flow of the program, but about the data flow. Condition coverage asks: the values of which boolean expressions have been covered? Multiple condition coverage is then interested in the combinations of boolean subexpressions within one decision operator. For the purpose of calculating this coverage metric, it can be sensible to disregard short-circuiting behaviour.

So in a condition without short-circuiting boolean operators or when disregarding short-circuiting behaviour, then it is clear that full path coverage for `if (a & b) ...` can be obtained with only two cases (e.g. `a=false, b=false` and `a=true, b=true`), whereas test cases are missing for full multiple condition coverage (here: `a=false, b=true` and `a=true, b=false`).

One practical benefit of using condition coverage is to show that the condition can actually influence the decision outcome. If not, we might unexpectedly have dead code. For this, multiple condition coverage is not necessary, but modified condition/decision coverage is sufficient. For the `if (a & b)` example, we would need three test cases for MC/DC: `a=true, b=true`; `a=true, b=false`; `a=false, b=true`. Therefore, path coverage doesn't imply MC/DC coverage either.

One problem using any kind of condition coverage is that this metric can be easily gamed by using control flow operators instead of logical operators. Additionally, incrementally building a decision like `bool d = true; d &= a; d &= b; if (d) ...` makes it non-obvious how `a` and `b` contribute to the decision. Therefore, condition coverage metrics are generally meaningless unless coupled to a specific coding style.

• @user970696 My point is that every short circuiting operator like `||` and `&&` is a branch point/control flow operator. To get full branch coverage, every `||` must branch both ways, i.e. be true and false. An optimizing compiler may sometimes be able to optimize a `||` to a `|`, but this is not generally the case. Branch coverage doesn't just consider constructs like `if` and `while` when calculated correctly, i.e. at the assembly level.
– amon
Jun 22, 2018 at 17:54