Code coverage tools work in two flavours:
- either the code is instrumented to record coverage statistics, or
- the program is run under a debugger or profiler, or tracing mechanism.
Coverage measurement is a dynamic quality assurance tool, as it measures which code is executed. Static analysis is not sufficient.
If a debugger is available, this makes it easy to interrupt normal execution after each statement and record that statement as covered. However, all these interruptions come at a noticeable runtime cost, which slows down your tests. Debugger-based coverage tools are often limited by the debugging interfaces in what kinds of coverage can be collected. E.g. you may not be able to collect branch coverage within expressions like
bar() && baz().
Instrumentation is performed by the compiler or a post-compilation step to inject code into the executable that records coverage. The source code is not modified. This has less runtime overhead than a debugger-based solution, but requires you to compile the program in a special coverage collection mode.
As an example, the Python
coverage.py tool uses Python's built-in tracing hooks. In contrast, GCC and Clang support instrumentation-based coverage collection when compiling with the
-fprofile-arcs -ftest-coverage flags (you should also disable optimizations and use a debugging build:
-g -O0). The advantage when collecting branch coverage: the compiler knows all branches that are present in the machine code, not just the branches easily visible in the source code. When the program is executed, it will record coverage in a file that can be massaged into reports with tools like gcov, lcov, gcovr, and many others. (Disclosure: I maintain gcovr.)
In general, coverage measurement needs the same kind of data as a profiler. Often, these tools use exactly the same infrastructure. However, a profiler can afford to be less exact since the hot spots will be executed often. Unlike coverage tools, they can use sampling to measure how often which code is executed. A sampling profiler regularly interrupts the process and collects a stack trace which points to the current location. This happens less often then at each statement, often only every few milliseconds. So they have less performance impact, but their data is less exact.