Unit testing statically typed functional code

Question

I wanted to ask you people, in which cases it makes sense to unit test statically typed functional code, as written in haskell, scala, ocaml, nemerle, f# or haXe (the last is what I am really interested in, but I wanted to tap into the knowledge of the bigger communities).

I ask this because from my understanding:

• One aspect of unit tests is to have the specs in runnable form. However when employing a declarative style, that directly maps the formalized specs to language semantics, is it even actually possible to express the specs in runnable form in a separate way, that adds value?

• The more obvious aspect of unit tests is to track down errors that cannot be revealed through static analysis. Given that type safe functional code is a good tool to code extremely close to what your static analyzer understands, it would seem that you can shift a lot of safety towards static analysis. However a simple mistake like using x instead of y (both being coordinates) in your code cannot be covered. OTOH such a mistake could also arise while writing the test code, so I am not sure whether it is worth the effort.

• Unit tests do introduce redundancy, which means that when requirements change, the code implementing them and the tests covering this code must both be changed. This overhead of course is about constant, so one could argue, that it doesn't really matter. In fact, in languages like Ruby it really doesn't compared to the benefits, but given how statically typed functional programming covers a lot of the ground unit tests are intended for, it feels like it's a constant overhead one can simply reduce without penalty.

From this I'd deduce that unit tests are somewhat obsolete in this programming style. Of course such a claim can only lead to religious wars, so let me boil this down to a simple question:

When you use such a programming style, to which extents do you use unit tests and why (what quality is it you hope to gain for your code)? Or the other way round: do you have criteria by which you can qualify a unit of statically typed functional code as covered by the static analyzer and hence without need for unit test coverage?

Answer 1

One aspect of unit tests is to have the specs in runnable form. However when employing a declarative style, that directly maps the formalized specs to language semantics, is it even actually possible to express the specs in runnable form in a separate way, that adds value?

If you have specs which can be mapped directly to function declarations - fine. But typically those are two completely different levels of abstractions. Unit tests are intended to test single pieces of code, written as white box tests by the same developer who is working on the function. Specs normally look like "when I enter this value here and press on this button, this and that should happen". Typically such a spec leads to far more than one function to be developed and tested.

However a simple mistake like using x instead of y (both being coordinates) in your code cannot be covered. However such a mistake could also arise while writing the test code, so I am not sure whether it is worth the effort.

Your misconception is here that unit tests are actually for finding bugs in your code first hand - that's not true, at least, it is only partially true. They are made to prevent you from introducing bugs later when your code evolves. So when you first had your function tested and your unit test working (with "x" and "y" properly in place), and then while refactoring you use x instead of y, then the unit test will show that up to you.

Unit tests do introduce redundancy, which means that when requirements change, the code implementing them and the tests covering this code must both be changed. This overhead of course is about constant, so one could argue, that it doesn't really matter. In fact, in languages like Ruby it really doesn't compared to the benefits, but given how statically typed functional programming covers a lot of the ground unit tests are intended for, it feels like it's a constant overhead one can simply reduce without penalty.

In engineering, most safety systems rely on redundancy. For example, two breaks in a car, a redundant parachute for a sky diver etc. The same idea applies to unit tests. Of course, having more code to change when the requirements change can be a disadvantage. So especially in unit tests it is important to keep them DRY (follow the "Dont Repeat Yourself" principle). In a statically typed language, you may have to write some less unit tests than in a weakly typed language. Especially "formal" tests may be not necessary - which is a good thing, since it gives you more time to work on the important unit tests which test substantial things. And don't think just because you static types, you don't need unit tests, there is still plenty of room to introduce bugs while refactoring.

Answer 2

One aspect of unit tests is to have the specs in runnable form. However when employing a declarative style, that directly maps the formalized specs to language semantics, is it even actually possible to express the specs in runnable form in a separate way, that adds value?

It's very unlikely that you will be able to completely express your specifications as type constraints.

When you use such a programming style, to which extents do you use unit tests and why (what quality is it you hope to gain for your code)?

In fact, one major benefit of this style is that pure functions are easier to unit-test: no need to setup external state or check it after execution.

Often the specification (or part of it) of a function can be expressed as properties relating returned value to arguments. In this case using QuickCheck (for Haskell) or ScalaCheck (for Scala) can let you write down these properties as expressions of the language and check that they hold for random inputs.

Answer 3

You can think of unit tests as an example of how to use the code, together with a description of why it's valuable.

Here's an example, in which Uncle Bob was kind enough to pair with me on John Conway's "Game of Life". I think it's an excellent exercise for this kind of thing. Most of the tests are full-system, testing the entire game, but that first one tests just one function - the one which calculates the neighbours around a cell. You can see that all the tests are written in declarative form, with the behavior we're looking for clearly spelled out.

It's also possible to mock functions used in functions; either by passing them into the function (the equivalent of dependency injection), or with frameworks like Brian Marick's Midje.

Answer 4

Yes, the unit tests already have sense with statically typed functional code. A simple example:

prop_encode a = (decode . encode $ a) == a

You can force prop_encode with static type.