# Path coverage and decomposition

Consider unit testing for a program written in procedural style. Each unit test tests a single function.

Consider the following example, in which I use `C_i(x)` for a condition depending on the argument `x` and `S_i` for a statement.

``````def foo(x):
if C_1(x): S_1
if C_2(x): S_2
if C_3(x): S_3
if C_4(x): S_4
``````

Assuming that all combinations of True and False values for the conditions can be achieved by passing different values of `x`, there are 2^4 possible execution paths and so 2^4 test cases are needed to achieve perfect path coverage.

Now, suppose we have decomposed this function:

``````def f_1(x):
if C_1(x): S_1

def f_2(x):
if C_2(x): S_2

def f_3(x):
if C_3(x): S_3

def f_4(x):
if C_4(x): S_4

def foo(x):
f_1(x)
f_2(x)
f_3(x)
f_4(x)
``````

Now we have five units to test. For each of `f_i`, two test cases are needed. For `foo`, there is only one execution path. In all, we have 2*4 + 1 test cases.

So, is it true that decomposition can make it that perfect path coverage can be achieved with a polynomial number of test cases instead of an exponential number?

• You are making the assumption that none of your `f_<n>` functions can possibly affect the result of any of the others. Commented Nov 26, 2022 at 17:23
• @PhilipKendall I understand what you mean. However, I am asking about the formal definition of the path coverage metric. Formally speaking, isn't there only one execution path in `foo` after decomposition (even if f_<n> are not independent)? Commented Nov 26, 2022 at 18:04
• It is important to understand that most of these metrics were defined before there were subroutines, modules, objects, classes, traits, interfaces, protocols, or any of this new-fangled nonsense. Just good old conditional GOTO. Commented Nov 26, 2022 at 18:54
• @JörgWMittag However, these metrics are used for unit testing and, to have unit testing, you need to have units in the first place. Commented Nov 26, 2022 at 19:04
• The answer is "You are correct". Not really a statement worth to be posted as a real answer. Commented Nov 26, 2022 at 21:08

Path coverage and code complexity measurements are metrics, and metrics can be gamed. Yes, you've correctly understood how a simple extract-function refactoring can drastically reduce the effort needed for achieving perfect coverage.

Some assorted thoughts related to this matter:

• Assuming that the extracted functions represent meaningful actions in the problem domain and are truly independent, you may have actually improved this code.

• I think testing private helper functions is a great approach, precisely because it can help keep the system under test smaller and more manageable. If you had to test the entire code only through the `foo()` function, then both variants of the code would require equivalent testing effort.

• While path coverage can be a great metric as a white-box test design technique, such metrics aren't necessarily useful in practice. For a lot of projects even 80% line coverage is good, and aiming for 100% path coverage would be excessive.

Coverage-guided test design can also lead to an overly mechanistic approach, resulting in tests that have low value. The purpose of tests isn't to achieve coverage, but to demonstrate that the software under test provides value to stakeholders, and secondarily to serve as a debugging tool. My current experience is that BDD-style test on the level of use cases have most value (“inverted test pyramid”). Coverage-guided approaches can be useful though for small fragments of very tricky code.

For example, I once used such test design techniques to comprehensively test a number parsing routine in a banking software. To my horror (but luckily well before the software went live), this led me to tests that showed how some number formats weren't recognized correctly, potentially corrupting million-dollar currency values. But that was the 1% tricky code in the software, whereas the main business value was provided by other parts. Nowadays, I would leave such coverage-guided testing efforts to automated fuzzers, though.

• "For a lot of projects even 80% line coverage is good". What would be an example of a software for public (rather than internal) consumption for which a buggy feature would not spell shame for the company that developed the software? Furthermore, if there are 20% of lines that are OK to go without testing, then maybe those lines should not have been written in the first place? Commented Nov 28, 2022 at 8:30
• "Nowadays, I would leave such coverage-guided testing efforts to automated fuzzers, though." Isn't it true that fuzz testing can only tell you whether the program crashed in response to an unexpected input, but not whether the output is correct if no crash occurred? Commented Nov 28, 2022 at 8:36