4

I've heard that it's ideal to have one assert (or expect or should, etc) per aspect of a component's behavior you're trying to test. Is this how it works? An addOne function would be tested like this, I suppose (example is written in javascript syntax):

it("returns one plus its input", function() {
  assert(addOne(5) == 6);
});

And I suppose that would be the complete extent of the tests for the addOne function, as each individual assert should be orthogonal.

How would one go about testing something more complex, but which still really has only one defining behavior?

Say for example we have an OO class for storing a chessboard, and each tile of the board is assigned an id number between 0 and 63. Then let's say there's a method getNeighbors which takes an integer from 0 to 63 and returns a sorted integer list/array/set containing the ids of all neighboring tiles. For example, if the numbers were laid out starting at 0 in the upper-left and incrementing across rows, then getNeighbors(0) would return [1, 8, 9] I suppose.

How many test cases would be written for this? I see two main options:

Test the entire behavior once

it("throws an error when called with a number below 0", ...);
it("throws an error when called with a number above 63", ...);
it("returns a list of the IDs of the neighboring tiles", function() {
  assert(getNeighbors(0) == [1, 8, 9]);
});

This seems like too weak of a test.

Test each case of the behavior once

it("throws an error when called with a number below 0", ...);
it("throws an error when called with a number above 63", ...);
it("returns [1, 8, 9] when called on 0", function() {
  assert(getNeighbors(0) == [1, 8, 9]);
});
it("returns [0, 2, 8, 9, 10] when called on 1", function() {
  assert(getNeighbors(1) == [0, 2, 8, 9, 10]);
});
... // and so on, for 62 more test cases

This seems over-the-top.

  • 1
    You're making a guess about the most likely places that bugs would exist. In this case, I'd say "the different types of boundary conditions." So, a test for each corner, a test for each "edge" (choose 1 square from each), and a test for an interior square. ofc, it's possible some implementation fails on some specific random interior square, or a different edge square than the one you tested, but i think this isn't nearly as likely as boundary condition bugs. – Jonah Jun 24 '16 at 20:30
  • 1
    If it is fast enough, a brute force enumeration of the entire problem space is always an option. – Frank Hileman Jun 24 '16 at 20:46
  • Possible duplicate of How should I unit test mathematical formulae? – gnat Jun 24 '16 at 20:56
8

You can do two things:

First, use parameterized tests to minimize the duplication of the test code:

   cases([
     [0, [1, 8, 9]],
     [1, [0, 2, 8, 9, 10]],
     // more testcases here
   ])
   .it('sample', function(n, expected) {
     expect(getNeighbors(n)).toEqual(expected);
   });

Second, partition your testcases into equivalence classes where you expect the code to behave exactly the same, and have only one test for each class. In your cases, the classes would probably be:

  • top left corner square
  • top right corner square
  • bottom left corner square
  • bottom right corner square
  • non-corner top border squares
  • non-corner left border squares
  • non-corner right border squares
  • non-corner bottom border squares
  • all inner squares
2

Say it's broken. Say you're thinking about something else entirely and don't have a lot of patience for this bug. Which test do you wish had been written now?

I test as much as I can get away with. At some point you have to stop and get paid.

Keeping that in mind, it's not simply behavor. It's the boundaries. Your behavor is 'show me the neighbors'. Your boundarys where the logic of that behavor changes are at 0,1,7, 8, 9, 16, 57, 58, & 63. These generate every shape posible around your square.

That's 9 tests I personally would rather see done in your first style before breaking them down for your second style. It's better to catch more and report it sloppy than to report exactly and miss much.

Ideally, If I was debugging your code (and some day I might be) I'd want all 9 in the second style. But in 20 years of coding I've yet to see 'ideal' code.

Please give me as much as you can.

1

tldr; The first "weak" version implies you wrote the entire implementation to pass the one test. You should use each of the brute force tests to drive your implementation, one at a time.

Okay the long version.

You tagged this with #TDD so I'll answer how I'd do this in a TDD based way. Generally you want to start with the simplest cases first and build your way up one at a time, aka the brute force approach. So your first two cases are pretty good, as they would result in very simple code to be written. However the way you explained your entire behavior once approach one it's clear you imagine writing that test and then implementing the entire algorithm, or writing the entire algorithm at first and then adding the unit tests. This isn't sufficient in either a test-driven or test-after approach.

After making each of your first two test cases pass I'd have no implementation yet - it would just be bounds checking. Now I'd look for the next test case I need to write the simplest code. I don't want to write the entire implementation yet - that's likely to have mistakes for multiple test cases at the same time. Instead what is one simple case that will force me to write more code?

CandiedOrange gave a good list of boundary cases, but 0 isn't a good first case. The reason being that 0 doesn't have anything above it or to the left of it. Those are special cases. Instead pick a case that's in the middle and make sure it spits out all the squares. That should be simple (no ifs or loops, just getting all the squares around it) and the most common setup. Then add one that is in the top row, the right row, the bottom row, the left row, and finally a corner or two.

Is that a perfect order? I dunno - perfect order doesn't exist - but it's probably close enough. After each test write just enough implementation to get the new test (and all previously written tests) passing. This means you should be able to focus on one case at a time, instead of debugging several at the same time.

At the end you might want to clean up the tests using a data driven approach. I rarely use those personally, but I do appreciate them when done well.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.