Suppose we have a service that returns users in certain categories, e.g.
- Listens to a certain artist.
- Listens to a certain playlist.
- Listens to a certain genre.
- Is between ages X and Y.
- Is located in geographic area Z.
Suppose an external
CategoryService manages valid categories, and we use a
CategoryServiceFixture for our integration testing. This fixture may or may not come with categories defined, e.g. we may use a fixture preloaded with actual production categories, or we may reset the fixture and add our own categories, e.g.
categoryServiceFixture.addArtist("Eminem"); categoryServiceFixture.addArtist("Red Hot Chili Peppers"); categoryServiceFixture.addPlaylist("Techno Bunker"); categoryServiceFixture.addGenre("Jazz"); categoryServiceFixture.addAgeGroup("90s Teens", 1970, 1990); categoryServiceFixture.addCountry("US"); categoryServiceFixture.addCity("New York");
Again, our service,
ListenerTargetingService, fetches users who match a certain filter, e.g.
listenerTargetingService.fetchUsers( new ListenerTargetingFilterBuilder() .addGenres("House", "Tech House") .addCountries("GB") .build());
I don't think it's necessary to write tests for every permutation of dimensions. I think it's sufficient to write tests such as...
@Test test_findByArtist_noneFound(); @Test test_findByArtist_someFound(); @Test test_findByArtist_invalidArtist(); @Test test_findByArtists_allFound(); @Test test_findByArtists_invalidArtist();
...for each dimension, and then a set of tests that combines filters:
@Test test_findByAllCategories_exactlyOneFound(); @Test test_findByAllButOneCategory_exactlyNFound(); @Test test_findByAllButTwoCategories_exactlyNFound(); // Doesn't need to be exhaustive. This is probably enough.
So here's where I got to thinking: Okay, so I can programmatically generate seed data so that the number of permutations found in each of the combined cases is always computable.
I can go ahead on my own, but it seems there would already exist a well-defined pattern or idiom for this kind of testing. What is it? Searching things like "permutation testing" mostly leads to results about methods in statistics, or papers on "orthogonal testing methods" (which are on target but too academic).
In case a package or tool can be recommended, I'm specifically using Java, JUnit, Mockito, Dagger, etc.