Does this unit test violate DAMP?

Question

In writing a split function that breaks up a sentence into each individual word based upon punctuation i.e. splits on something like the character class [\s,.?!]+, though for the purposes of this question the implementation is not important, and the list of characters to split on should be considered to be of arbitrary length. This question is specifically about designing good unit tests.

I found myself writing my (ExUnit) tests in this way:

defmodule MyModuleTest do
  use ExUnit.Case, async: true
  doctest MyModule

 test "splits on '?'" do
    assert MyModule.split("a" <> "?" <> "a")               == ["a", "a"]
    assert MyModule.split("a" <> "?" <> "?" <> "a")        == ["a", "a"] ## Multiple chars
    assert MyModule.split("?" <> "a" <> "?" <> "a" <> "?") == ["a", "a"] ## Padded string
  end

  test "splits on '!'" do
    assert MyModule.split("a" <> "!" <> "a")               == ["a", "a"]
    assert MyModule.split("a" <> "!" <> "!" <> "a")        == ["a", "a"] ## Multiple chars
    assert MyModule.split("!" <> "a" <> "!" <> "a" <> "!") == ["a", "a"] ## Padded string
  end

  test "MyModule.splits on '.'" do
    assert MyModule.split("a" <> "." <> "a")               == ["a", "a"]
    assert MyModule.split("a" <> "." <> "." <> "a")        == ["a", "a"] ## Multiple chars
    assert MyModule.split("." <> "a" <> "." <> "a" <> ".") == ["a", "a"] ## Padded string
  end
#   ## etc. etc.

end

Which could of course be DRY'd up by doing something such as the following:

 Enum.each ~w(. , ? ! \s), fn char ->
    @char char
    test "splits on \"#{@char}\"" do
      assert MyModule.split("a" <> @char <> "a")                   == ["a", "a"]
      assert MyModule.split("a" <> @char <> @char <> "a")          == ["a", "a"] ## Multiple chars
      assert MyModule.split(@char <> "a" <> @char <> "a" <> @char) == ["a", "a"] ## Padded string
    end
  end

However, certain schools of thought posit that when unit testing, tests should be DAMP and not DRY.

• In this case, is the second example an anti-pattern?
• If so, is the first (or another) way a better approach to structuring this test?

EDIT: While feedback on this specific case is appreciated, this question is primarily about software testing patterns. This question was asked in order to obtain information or thoughts concerning whether looping over a list (of arbitrary data or length) in order to generate test cases violates the principle of DAMP unit tests.

Answer 1

I find the first much preferable. Here are the issues I see with the second version:

• It's harder to tell what you are testing.
• I think maybe it will be more difficult to know what exactly went wrong in a failed test.
• It could use tests of its own which I think is a problem.
• When requirements change e.g. use cases diverge, this kind of thing can be difficult to untangle.

In my book: the dumber the tests the better. The more fancy you get with tests the more likely you are going to build a flaw into the test. There are good reasons to use loops and such but I would err on the side of being more explicit.

Answer 2

Your second example is certainly not an anti-pattern. In fact, there are testing frameworks that support exactly that kind of tests: You write a single testcase that takes some parameters and next to that you specify a set of values for those parameters and the expected outcomes.

As your testcases are so similar, they only differ in which character for splitting is used, the second approach is the clearest in my view. It could be made even clearer if you would use the facilities of your testing framework for parametrized tests.

Answer 3

If you follow TDD where you write a failing test, fix that, and repeat, you tests would come out different.

(Note: I start out with degenerate cases and then slowly add the next easiest functionality to test and implement)

Here is a good test design:

1. first test an empty string
2. test you don't split a string without the control characters
3. Then write a test for "a" <> @char <> "a" with a ?
4. Then write a test for "a" <> @char <> "a" with a !
5. Then write a test for "a" <> @char <> "a" with a .
6. Then tests for double control characters and padded and mix of multiple types of control chars etc.

Now each test is really small, concise and readable so when you break things, it should be immediately obvious when you run your tests.