1

I've just written a simple mathematical library which consists of pure functions that take a few arguments, do some computation and return a result. I'd like to write unit tests for this library, but most frameworks I could find (Jasmine, Mocha, QUnit, etc.) include a wide range of bloat, ranging from regex matching to async calls. I don't need to do any of that - indeed, a list of assertions would work well for me:

const assert = require("assert");
assert(functionA(a0, b0, c0) === d0)
assert(functionA(a1, b1, c1) === d1)
assert(functionA(a2, b2, c2) === d2)
assert(functionB(x0, y0) === z0)
assert(functionB(x1, y1) === z1)
...

What benefit do I gain from using those huge frameworks rather than a simple list of assertions?

2
  • 2
    Why do you care if the unit test library contain functionality you don't need? Using an existing library is always less work and maintenance than writing your own, however lightweight.
    – JacquesB
    Oct 16, 2016 at 18:08
  • Uh, no using a library is not "always less work". Where did you get that certitude? If all you want is a couple of asserts that fail, then using a library potentially means having to go through loads of documentation to learn and understand how to install and use the functionality that you need. More functionality usually (but not necessarily) means more complexity, more documentation more things that can go wrong therefore more time spent fixing your library. Besides, bloat is a waste of storage, CPU cycles, time, and ultimately a waste of electricity therefore pollution.
    – Rolf
    Mar 14, 2018 at 1:33

2 Answers 2

6

The first purpose of a test suite is to verify that your code seems to work as expected.

But what if something goes wrong? The second purpose is to make it easy for the programmer to easily understand what went wrong, and where.

Better diagnostics

When a simple assertion fails, all we are told is that it failed – no reason why. Well, at least we are given the line number of the failing assertion. Example of an imaginary test framework:

ERROR: assertion failed
in test.js line 165:
assert(result === expected);

This makes it cumbersome to debug the problem – we have to look at the source code for the test, log the values before the assertion, re-run everything, hope that the problem is still present, and can then start hunting for the bug.

When the assertion function is not given a boolean but two values to compare, it can provide much better error messages:

ERROR: assertion failed
in test.js line 165:
assertEquals(result, expected);

got:      [42, 42, 42]
expected: [41, 42, 43]

Often, we don't even have to look at the test any more and can immediately understand the problem from the logged values.

Since equality is not the only relation that might be tested, we now need various assert* functions. E.g. assertNotEquals(result, otherValue), assertThrows(function), assertMatches(result, expectedRegex), ….

Better test organization

The next advantage of a test framework is that it helps us organize our tests, and makes it easy to process the test results.

A single test case that tests everything is problematic, because that makes it difficult to discover the tests for a particular module or function in our source code. I prefer an RSpec-style describe/it organization:

describe("functionA", () => {
  it("sums the values", () => {
    assertEquals(functionA(2, 3, 4), 9);
  });

  it("throws for negative numbers", () => {
    assertThrows(() => functionA(2, 3, -1));
  });
});

An error message can now clearly explain what we were testing:

test "functionA throws for negative numbers" failed:
did not throw
in test.js line 165:
    assertThrows(() => functionA(2, 3, -1));

Better test reporting

Even more importantly, the test runner is now responsible for executing the individual test functions. If one assertion fails, unrelated tests will still be executed. This makes it possible to tell how many tests failed in total, e.g. “FAILED 3 of 367 tests” or “PASSED all 367 tests”. This helps you be certain that all your tests did actually run.

Some test runners make their output more clear by only displaying details of failed test cases, and by highlighting important parts with colours.

Also, a test runner can write the test results in a machine-readable format, e.g. TAP or XML. These can then be visualized and processed by IDEs or continuous integration tools.

0

If you do unit testing properly the huge benefit you'll get is being able to remove testing from your library. You don't have to test that functionA works every time you run functionA. You have to test it every time you rewrite it.

Input validating tests will still need to be done at run time but checking that methods give the correct result doesn't need to be done within those methods. That code can be moved to elsewhere. Then your users will never see or run this test code. Do that and nothing other than your development environment even knows what framework you're using.

Some frameworks will invite you to become dependant on them because that's good for their business. Not so much for yours. Feel free to treat them like a library. Use only what you need only where you need it.

1
  • 3
    It's not hard to whip up a test runner by hand. I don't think OP meant they were going to put the asserts inside the CUT.
    – RubberDuck
    Oct 16, 2016 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.