Say I am a QA person so I have no control over the complexity of the code in the following example.

I have a datatable widget with many different filtering and searching options. All the columns on my data table are also sortable.

There are many issues with components not working together. For example, if someone uses Filter A & Filter B together, it will break. Or, if someone uses Filter A and sorts by Column A, it will break.

One more example. If I have 3 columns, I want to test with column 1 active, column 1 & 2 active, column 1,2, & 3 active, column 2 & 3 active and so on.

How do I determine all of the possible states of the widget? If I have 5 columns and 6 filters, and the user can sort by as many columns and filters they want, how do I generate a list of the combinations I need to test?

  • 3
    Are you looking for how to calculate combinations and/or permutations? en.wikipedia.org/wiki/Combination vs en.wikipedia.org/wiki/Permutation for instance? If you can have as many filters by as many columns as wanted, you'll end up with the inclusion of a factorial, so the amount of possible states will be "very large" - probably so that you'll need to set limits on what states are tested as it is infeasible to test them all. – BrianH Jun 26 at 19:39

You are looking at a combinatorial problem:

  1. You have 3 columns, so you need all combinations of 3 columns: [[],[1],[2],[1,2],[3],[1,3],[2,3],[1,2,3]] where [ and ] enclose a list of included columns. And yes, [] is a valid combination.
  2. You have 2 filters available for each column, plus an unfiltered state. You also have ensure that you have data that represents at least two values for each of the filtered and unfiltered settings.
  3. You have 4 sorted states for each column: never sorted, previously sorted, sorted up, sorted down. This combination will be used to test for stable sorting processes.

How do you generate all of these tests? I would be tempted to generate the combinations using a language that handles combinatorics readily. I would be tempted to use Haskell for this purpose, but Python should be able to do a passable job. Your generator would create the test code; you would then vette the test code and need to maintain it.


Your 5 columns and 6 filters example is simply a nested loop.

for (int column = 0; column < numberOfColumns; column++)
    for (int filter = 0; filter < numberOfFilters; filter++)
        RunTestForColumnFilterCombination(column, filter);

To use all bit combinations in a range:

int getBit(int bits, int position)
    return ((bits >> position)  & 0x01);

for (int bits= 0; bits<= 7; bits++)
    RunTestForColumns(getBit(bits, 0), getBit(bits, 1), getBit(bits, 2));
  • No, I want to test with all possible combinations. If I have 3 columns, I want to test with column 1 active, column 1 & 2 active, column 1,2, & 3 active, column 2&3 active, and so on. Thank you for your answer though – CamelBlues Jun 27 at 17:18
  • Those are just bits. For three columns, loop through the numbers zero through seven; each bit is a column "active" flag. – Robert Harvey Jun 27 at 20:00
  • @RobertHarvey just a question. Does QA must code tests such like this? Aren't these test unit tests? If QA needs to code unit tests without specifications, something is going wrong in the development. – Laiv Jun 28 at 8:33
  • 1
    @Laiv: No. It's just a way to create all of the combinations required. – Robert Harvey Jun 28 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.