When writing unit tests, I feel that there is a trade-off between code repetition and test logic.
Example of my current (likely flawed) approach:
To test this function (overly simple function for illustration):
from warnings import warn
class AmbiguousSignWarning(Warning):
pass
def product_sign(arg1, arg2):
"""
Returns 1 if product is non-negative and -1 if negative. Warns when product is 0.
"""
product = arg1 * arg2
if product >= 0:
if product == 0:
warn(AmbiguousSignWarning(
"0 can be considered both positive or negative. Treated as positive."
))
return 1
else:
return -1
I'd write something like:
from warnings import filterwarnings, resetwarnings
from pytest import mark, warns
PRODUCT_SIGN_TEST_CASES = [
{
"description": "arg1 > 0, arg2 > 0",
"arg1": 1,
"arg2": 14,
"expected_warning": None,
"expected_result": 1,
},
{
"description": "arg1 == 0, arg2 > 0",
"arg1": 0,
"arg2": 14,
"expected_warning": AmbiguousSignWarning,
"expected_result": 1,
},
{
"description": "arg1 < 0, arg2 > 0",
"arg1": -12,
"arg2": 14,
"expected_warning": None,
"expected_result": -1,
},
# Goes on like this to exhaust combinations of signs of arg1 and arg2
]
class TestProductSign:
@mark.parametrize("test_case", PRODUCT_SIGN_TEST_CASES)
def test_product_sign(self, test_case):
if test_case["expected_warning"] is None:
filterwarnings("error", category=AmbiguousSignWarning)
result = product_sign(
arg1=test_case["arg1"],
arg2=test_case["arg2"],
)
resetwarnings()
else:
with warns(AmbiguousSignWarning):
result = product_sign(
arg1=test_case["arg1"],
arg2=test_case["arg2"],
)
assert result == test_case["expected_result"]
What I don't like about this approach is that I basically copy paste test cases, modifying them slightly depending on what each tests. Copy-pasting is a no-no (modifying the function would likely end up in having to modify each test case), but how do I avoid code duplication without introducing more test logic?
More logic means more potential for error (perhaps not in this simple example, but in more complex functions); at some point, I would have to test my test logic. My approach also bloats up the testing module as functions that have more complex signatures, or arguments end up using a lot of lines of code just to define the test cases.
If this indeed is an unavoidable tradeoff, should I prefer duplicated (logic-free) code (higher maintenance) over more testing logic (sources for error), especially in a setting where correctness of code is vital?
Some comments on why this design:
- I wanted to avoid logic as much as possible: single test with minimal logic tests all cases; developer doesn't need to understand the logic of multiple tests
- The code that is duplicated basically contains no logic, so downstream modifications may be laborious, but less error-prone
- All test cases are written as literals; minimal code to arrange test cases
- Addition and removal of test cases should require only looking at the variable
PRODUCT_SIGN_TEST_CASES, seeing what dictionary key the other tests provide, and mimicking it. - A test_case object is usually printed by pytest when it fails, so that inspection of "description" explains what is tested (since I don't have descriptive test name)
Potential antipatterns I tried to avoid:
- Lots of testing code, requiring someone who reads the code to understand various test structures
- Need for testing unit test code due to complex logic
I couldn't find my answer here:
- Is it OK to repeat code for unit tests?
- Is it bad practice to repeat logic being tested in unit tests?
- Is having some logic in source code in order to perform some tests a good practice?
Also, I looked at some repositories on GitHub and found them either to have too much testing logic for my taste, or repeated testing logic achieving the same as I do with repetition in the PRODUCT_SIGN_TEST_CASES variable (example: https://github.com/scipy/scipy/blob/main/scipy/linalg/tests/test_decomp_cholesky.py).
