I have the following problem:

  • I would like to test complex business logic for each test case completely separately, i.e. all tests should run in parallel. I don't want the test for customer #43 to depend on customers #1 through #42 passing, or having to comment them out.
  • I would like to test that I don't accidentally mess up aggregate functions

However, it seems these two are philosophically at odds. Consider the example below (in Python/Pandas). In my example. In the example groupby("A") only has an actual impact if more than 1 customer is tested at once. So essentially I could have the business logic pass for all customers while testing, but fail "silently" because it doesn't actually group as it should.

My options seem to be either

  • Running the entire dataframe in a single test, and assert in serial, or
  • Running the test in parallel, knowing that aggregate logic is not properly tested, and having to write a separate test to assert on e.g. the sum of the output. However, this gets slightly more weird when working with dates (do I start summing seconds since 1970?). I also feel this is perhaps a bit testing for testing's sake, since I don't really care about the aggregate outcome, as much as the individual.

Does a general pattern exist, or am I testing the wrong thing?

import pandas as pd

df = pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]})

# Some business logic to test
def logic(df):
    return df.groupby("A", as_index=False).sum()

# The test
for _, customer in df.iterrows():
    # Setup for example's sake
    customer = customer.to_frame().T
    result = logic(customer)["A"].item()
    expected = customer["A"].item()
    assert result == expected
  • 1
    I think you're misunderstanding terms. Aggregate functions by definition act on more than one record. But that doesn't mean that they constitute more than one "test case" that should somehow be kept separate. May 3, 2022 at 9:06

1 Answer 1


More generally what you are describing is a Data-Driven Test. To echo Kilian Foth's comment, each data frame is not a separate test case. Each data frame represents a different input to the same test case.

How you implement a data-driven test depends on the unit testing framework being used. Some frameworks have an explicit API for data driven tests. Frameworks without a specific API can still do data-driven testing. Basically it involves just what you did. Define an array of data representing the test inputs. Loop over that data and perform your test inside the loop.

The challenge is providing isolated inputs. If your dataframe needs to aggregate information, each dataframe used as input to your test should be a complete dataframe capable of being aggregated without affecting other test inputs.

In your case, if grouping by A is part of the logic being tested, you might need to create multiple data frames. Each one has an A and a B to ensure things are grouped properly:

frames = [
  pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]}),
  pd.DataFrame({"A": [2, 3, 4], "B": [5, 6, 7]}),
  pd.DataFrame({"A": [5, 6, 7], "B": [8, 9, 10]})

def test()
  for df in frames:
    for _, customer in df.iterrows():
      # arrange
      # act
      # assert

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.