How should I test "Glue Functions" without testing that "the code I wrote is the code I wrote"?

Question

I usually write my code in a test driven style. I write tests as specifications and then my code. It's great and useful.

I always try to ignore implementation when testing and only test behaviour. I don't care how it gets done, just that it got done. I find this especially easy for functional programming.

Now here is the problem I have found. I have an app that is written in a functional style. All of the unit tests are nice, clean, behavioural tests. I only ever check output and don't do things like "did you call this function?".

At some point however, I start needing "glue" functions. I'll consider these functions that don't introduce a lot of functionality and largely just call my other existing functions. Perhaps chaining a bunch together or whatever feature it may be.

How do I test these glue functions?

I ask because I want to avoid two main things as much as possible:

• I don't want to test them by simply mocking what they do and seeing if specific functions were called.
• They have a desired output I want, but usually this output is just a series of outputs from other functions that are already tested. I don't want to repeat myself and just "re-test" those inner functions to see if my glue function called them.

Hopefully that makes sense. Here is an example (written in pseudo code):

func1 (x) => x + 1;
func2 (x) => x * 2;

glue (x) => [func1(x), func2(x)];

Here would be a simple way of testing these functions.

testFunc1 () => expect func1(2) == 3;
testFunc2 () => expect func1(2) == 4;

testGlue () => expect glue(2) == [3, 4];

So obviously, glue has an expected and predictable behaviour I want to model. I know that in this example these tests might be ok. So consider instead that the outputs of func1 and func2 are not simple numbers but much more complicated objects.

In such a case, implementing the checks that glue output the correct objects would be tedious AND it would be totally duplicated from the individual tests of func1 and func2. This also leads into the next issue.

Instead consider:

testGlue () => expect glue(2) == [func1(2), func2(2)];

This certainly seems better. But I think it is still flawed. While this means I am not repeating my test code it now instead "tests that the code you wrote is the code you wrote" (as opposed to what the behaviour is). Again, in such a small example it's not an issue so pretend that within glue a few variables are swapped around and yada yada is done so that to test it in this way would require my test to also set up the variables like such. Then we would be basically copying the code from the function to check if func1 and func2 were called with the correct variables leading to repetition and testing that "it's the code you wrote".

If a larger example is needed to showcase such results just let me know and I will get one. Hopefully there is some good discussion to be had here.

I anticipate someone to answer "don't use glue functions" and to that I preemptively ask, "what's the alternative method?".

EDIT:

So I am beginning to think that an alternate question that would also give me the answer I want is this.

Consider that the output of func1 and func2 is something too big to feasibly have as a hardcoded value in the test. Maybe it's an object or something.

Does writing my test of glue as:

testGlue () => expect glue(2) == [func1(2), func2(2)];

No hold on, I must clarify something. Obviously the above test is absolutely stupid. It is just "the code I wrote is the code I wrote". We MUST imagine that the function does more than this. In a real world scenario glue would do some processing of x before passing it around. The order of the array might matter. And however many other options. So maybe I'm checking that glue(2)[0] = func1(3) instead (pretend there is further processing to it).

In such a case, is it still considered bad practice to use the output of a function as something to test against (even though that function is tested somewhere else)?

Answer 1

Boring structural code doesn’t need isolated testing.

Test interesting code. That code has a behavior. Nail down the behavior you expect and not only will your code likely be correct, it’ll be easier to read.

But keep that interesting code away from the boring structural code. Do that and I’ll be able to read it and trust it without a test that isolates it.

Now if the boring structural code is part of a chain of integrated peripherals and behavior objects then fine, throw an integration test at it. If you’d like to test if I’m full of it, break the structural code and see how long it takes someone to find the problem.

Don’t waste time solving non problems. At best you’re only amusing yourself. At worst you’re actually making it harder to refactor the code.

Remember: it’s not that every function needs a test. It’s every behavior.

Answer 2

This is normal. you should write:

testGlue () => expect glue(2) == [3, 4];

You worry that because you have already tested the component functions you don't need this test, but that is untrue.

The glue function is called, the caller doesn't care how its implemented, but does expect a result. Later you might change it to use func4 and func5 but the expected output should be the same.

If anything its the func1 and func2 tests which are redundant. Consider if instead of composing glue from other function you simply wrote:

glue(x) => [x+1,x+2]

or lets make it a bit more real life

BasketInvoice(items) => [
    items.map(i.cost)..., 
    sum(items.map(i.cost) * tax, 
    sum(items.map(i.cost) + sum(items.map(i.cost) * tax
  ]

Now you just have the one function and the one test:

BasketInvoiceText => expect  BasketInvoice(testItems) == [1,2,3,0.5,6.5]

You then refactor the code to remove the duplication but don't expose these private functions. Do you need to test them? I mean it might help you code and bugfix, but its not an exposed interface, its just an implementation detail

TDD makes you write a lot of tests, that's its thing, don't worry about writing too many tests just go with the flow.

Answer 3

How should I test “Glue Functions” without testing that “the code I wrote is the code I wrote”?

This is the easiest part of your question. I refer you to C. A. R. Hoare

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult

If you can reduce the complexity of your glue code to the point where there are obviously no deficiencies, then "review by a human being" can be your primary testing strategy.

This is especially true when the glue code is stable - either these requirements don't change, or we have the discipline to write new glue code when new requirements appear.

See also Rich Hickey Spec-ulation.

The calculus of testing involves risk management and investment. Test automation pays an upfront cost for development, and a small tax each time the test is run. You win if the test catches enough mistakes, where enough is an expression of the sum of the damages of the mistakes over the effective lifetime of the test.

For code that is unchanging, that you got right the first time, the investment odds aren't so great.

glue(x) => [func1, func2].map(f -> f(x))

Once you are here, where are errors going to appear? The only place they could show up is in the ordering of the functions in the list. How often are you changing that?

Design is what we do to get more of what we want than we would get by just doing it -- Ruth Malan

Isolating high risk and low risk code is a potential "what we want"; and we should be alert to the fact that the investment odds for the two cases are very different.

The problem that we face - especially early in our designs - is that the boundaries between high and low risk code are not immediately obvious, and we often end up coupling mixed risk levels together.

---

In TDD, what I think happens is something like this: you'll start with a guess as to where you should be testing, but as you refactor you'll start teasing apart the low and high risk parts of the design. The high risk modules will continue to acquire more tests; the tests that aren't directly measuring high risk code are eventually pruned away.

---

In such a case, is it still considered bad practice to use the output of a function as something to test against (even though that function is tested somewhere else)?

No; it is perfectly reasonable to have automated checks that A(x,y,z) "behaves like" B(x) for certain combinations of x, y, and z.

You do want to pay careful attention to the value of those tests - how often are they detecting errors that you don't detect elsewhere, etc.

Answer 4

I find it hard to discuss such super abstract issues, so let me transform your example into a practical one:

• func1 shall be a complicated (but pure) function, calculating a value from some input.
• func2 the same.
• glue is a "stupid" function which takes one parameter (a string which encodes the input values of func1 and func2 encoded as a structured JSON document) and returns a single string (again a JSON document) - in other words, a REST API.

If I were to implement tests for those, it would depend mostly on which of these functions are actually called from outside.

If I knew that 99% of all users will always only call glue, then my test would call glue. The test would not even know that func1 and func2 exist.

If, instead, the two functions were to be called often directly, and glue were just a boring add-on (a further, optional, way of calling them indirectly), then I would maybe start with writing tests for func*.

In both cases, I would write additional tests for the so far missing functions only if if were very simple/quickly done; if I had nothing better to do; or if there were other particular reasons to do it (for example, even if glue were boring and irrelevant, I might add a test for it if it were the only spot in my code where I use these JSON conversions; this would save me from later accidentally deleteing the JSON library from my code base, or detect breakages if that library introduced breaking changes).

In summary, there are no hard and fast rules for this; some of it comes down to taste, and to circumstances. Nothing keeps you from treating all three functions as a single unit (accessed through glue) if it makes sense in your particular case.

Answer 5

You may wish to simply put your edit in a new question since this one's already had it's ride as a hot network question. Until then, rather than update my answer I'll address your edit here since I'd rather see this answer voted on independently.

You wrote:

EDIT:

So I am beginning to think that an alternate question that would also give me the answer I want is this.

Consider that the output of func1 and func2 is something too big to feasibly have as a hardcoded value in the test. Maybe it's an object or something.

This a bit of a tangent, but I'm going to react to this "too big" idea here:

So serialize it in a file and make it a resource the test suite can load. This doesn't mean never crack it open and check that every part of it is correct. Do that. By hand if need be. But once you know what the correct form is you can save that and create a regression test that will complain when this changes.

Now sure that doesn't tell you which part changed. But why reinvent a test that amounts to a visual difference hex editer when those already exist? We have those tools already. Lets stop rebuilding them.

But fine, let's say it's "too big"...

Does writing my test of glue as:

testGlue () => expect glue(2) == [func1(2), func2(2)]; No hold on, I must clarify something. Obviously the above test is absolutely stupid. It is just "the code I wrote is the code I wrote". We MUST imagine that the function does more than this. In a real world scenario glue would do some processing of x before passing it around. The order of the array might matter. And however many other options. So maybe I'm checking that glue(2)[0] = func1(3) instead (pretend there is further processing to it).

In such a case, is it still considered bad practice to use the output of a function as something to test against (even though that function is tested somewhere else)?

Also testing a function somewhere else is a non-problem. If it wasn't we wouldn't have unit tests and integration tests on the same project. Many functions are part of functions. Many tests test parts that were already tested in a larger test. This is normal. It would only bother me if two tests were so identical that having them both adds nothing.

What bothers me here is isn't that glue() (however you complicate it to avoid my 'boring code' policy) and func1() both get tested. It's that tests test against an API. Each test should test against one API. I use the test to understand that API when I read it.

Now many API's have smaller API's inside. Large functions being supported by smaller functions. That's all fine. But it's a violation of the abstraction of the larger API to reach inside it and talk to it's smaller internal APIs. Not that the internal ones can't be tested. Please test them. On their own. But mixing abstractions like this means you're not really testing the API's behavior. You're testing it's structure. It's internal implementation. When you do that you are not enabling refactoring. You're actually setting your code in cement. You're making it very hard to refactor. We call it software because it's meant to be easy to change.

Think about the time you decide you want to implement glue differently. You may decide to delete func1() entirely. And if you keep good abstractions going in your code that wouldn't be difficult. But it would really stink to do that only to find a bunch of glue() tests failing because they happen to know that func1() used to exist.

Adding unit tests is supposed to make writing code easier. And it will. If you do it right. Please do it right. Because when it's done wrong it makes this so much harder.

Answer 6

Have you considered the failure scenarios?

(I'm assuming your glue logic has some kind of handling to deal with failures in the downstream dependencies, or guards to prevent invalid input from getting into the downstream functions)

Given your example,

func1 (x) => x + 1;
func2 (x) => x * 2;

glue (x) => [func1(x), func2(x)];

Test these inputs and does it do what it's supposed to?

• glue(null)
• glue("a")

Answer 7

I will only talk about unit tests here, not integration tests etc. which are necessary but irrelevant to this issue.

It is much simpler than you think.

An example is worth a thousand words, so imagine a function that can register a user.

This is a relatively complex process depending on the application you are developing, and it is often preferred to implement it in a function that will call other functions, and not do anything else.

So, like you, the first question that arises is "am I not just rewriting the code in a test? In reality, it's a question of perspective. And that's the most important thing here, you have to look up and see things from the highest possible level. For example, if you were to define in an abstract way the registration of a client in your application, you might say :

• add a record in the users table with the email and password
• call the mail service to send a confirmation mail to the customer
• we call the sms service to welcome him
• ... But this is a technical point of view, it is internal. It will change. So it is by definition wrong in the long run. A more abstract, higher-level view would be :
• the customer has an account with us
• we have notified the customer of the creation of his account

The higher the abstraction and the more product-oriented it is, the more the test will prove not that the function calls other functions, but that it fulfils its role correctly. Furthermore, the code written in this function can remain unchanged for much longer than if it technically described the inside of the function.

Answer 8

It depends on the semantics of the glue function.

If the purpose of the glue function is just produce f1(x) and f2(x) I would test that: glue(x) == [f1(x), f2(x)]. A numerical example I can think of would be f1=sin, f2=cos, glue = sin_cos` (i.e. return the pair). An initial implementation might be to just use the two functions, a more efficient implementation may not require using both functions. Note that the key invariant here is that the glue function reproduces the values of the two functions, however it is implemented.

Conversely if the glue function's role is to produce a pair of values, and those values happen to already have functions to compute them, then I would test that: glue(x) = [a, b]. Then, if one or both of the f functions get removed or renamed, at least the test won't have to change.

Answer 9

You can test the glue function using mocks if goal is not to perform an integration test of the combined behaviour of glue, func1 and func2 but to unit test that glue calls func1 and func2 and to be independent of func1 and func2's behaviour.

For example, the pseudo-code:

with mock(func1) as mock_func1, mock(func2) as mock_func2:
    mock_func1.expected_return = 3
    mock_func2.expected_return = 4

    expect glue(2) = [3, 4]
    expect mock_func1.times_called = 1
    expect mock_func1.called_with = [[2]]
    expect mock_func2.times_called = 1
    expect mock_func2.called_with = [[2]]

This divorces your unit test of the glue function from any functionality of func1 and func2 and tests that the glue function does what you expect (i.e. returns an array with the expected return values and calls each function exactly once and with the expected arguments).

This also means that if you change func1 then your test for glue will not fail. This is particularly important for a long chain of functions so that one small change does not impact a large number of unit tests that should be unrelated (you can still use integration tests along side unit tests if you do want to test that chain of functions).