8

Let's assume I wrote an extension method in C# for byte arrays which encodes them into hex strings, as follows:

public static class Extensions
{
    public static string ToHex(this byte[] binary)
    {
        const string chars = "0123456789abcdef";
        var resultBuilder = new StringBuilder();
        foreach(var b in binary)
        {
            resultBuilder.Append(chars[(b >> 4) & 0xf]).Append(chars[b & 0xf]);
        }
        return resultBuilder.ToString();
    }
}

I could test the method above using NUnit as follows:

[Test]
public void TestToHex_Works()
{
    var bytes = new byte[] { 0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef };
    Assert.AreEqual("0123456789abcdef", bytes.ToHex());
}

If I use the Extensions.ToHex inside my project, let's assume in Foo.Do method as follows:

public class Foo
{
    public bool Do(byte[] payload)
    {
        var data = "ES=" + payload.ToHex() + "ff";
        // ...
        return data.Length > 5;
    }
    // ...
}

Then all tests of Foo.Do will depend on the success of TestToHex_Works.

Using free functions in C++ the outcome will be the same: tests that test methods that use free functions will depend on the success of free function tests.

How can I handle such situations? Can I somehow resolve these test dependencies? Is there a better way to test the code snippets above?

  • 4
    Then all tests of Foo.Do will depend on the success of TestToHex_works -- So? You don't have classes that depend on the success of other classes? – Robert Harvey Jun 26 at 15:35
  • 5
    I've never quite understood this obsession with free/static functions and their so-called non-testability. If a free function is free of side-effects, it is the easiest thing on the planet to test and prove it works. You've demonstrated this quite effectively in your own question. How do you test ordinary, side-effect free methods (that aren't dependent on class state) in object instances? I know you have some of those. – Robert Harvey Jun 26 at 15:37
  • 2
    The only downside of this code using static functions is that you can’t easily use something else than toHex (or swap implementations). Apart from that everything is fine. Your code converting to hex is tested, now there’s another code using that tested code as a utility to achieve its own goal. – Steve Chamaillard Jun 26 at 16:15
  • 3
    I am completely missing what is the problem here. If ToHex doesn't work, then it's clear that Do won't work either. – Simon B Jun 26 at 16:17
  • 5
    The test for Foo.Do() shouldn't know or care that it calls ToHex() under the covers, that's an implementation detail. – 17 of 26 Jun 26 at 17:22
37

Then all tests of Foo.Do will depend on the success of TestToHex_Works.

Yes. That's why you have tests for TextToHex. If those tests pass, the function meets the spec defined in those tests. So Foo.Do can safely call it and not worry about it. It's covered already.

You could add an interface, make the method an instance method and inject it into Foo. Then you could mock TextToHex. But now you have to write a mock, which may function differently. So you'll need an "integration" test to bring the two together to ensure the parts really work together. What has that achieved other than making things more complex?

The idea that unit tests should test parts of your code in isolation from other parts is a fallacy. The "unit" in a unit test is an isolated unit of execution. If two tests can be run simultaneously without affecting each other, then they run in isolation and so are unit tests. Static functions that are fast, do not have a complex set up and have no side effects such as your example are therefore fine to use directly in unit tests. If you have code that is slow, complex to set up or has side effects, then mocks are useful. They should be avoided elsewhere though.

  • 2
    You make a pretty good argument for "test-first" development. Mocking exists in part because code gets written in such a way that it is too hard to test, and if you write your tests first, you force yourself to write code that is more easily testable. – Robert Harvey Jun 26 at 17:57
  • 3
    I'm upvoting purely for recommending not mocking pure functions. Seen that too many times. – Jared Smith Jun 27 at 2:03
2

Using free functions in C++ the outcome will be the same: tests that test methods that use free functions will depend on the success of free function tests.

How can I handle such situations? Can I somehow resolve these test dependencies? Is there a better way to test the code snippets above?

Well, I don't see dependency here. At least not the kind that forces us to execute one test before another. The dependency we build among tests (no matter the kind) is one confidence.

We build a piece of code (test-first or not) and we ensure tests pass. Then, we are in a position of building more code. All the code built upon this first is built upon confidence and certainty. This is more or less what @DavidArno explains (very well) in his answer.

Yes. That's why you have tests for X. If those tests pass, the function meets the spec defined in those tests. So Y can safely call it and not worry about it. It's covered already.

Unit tests should run in any order, any time, any environment and as fast as possible. Whether TestToHex_Works is executed the first or the last should not worry you.

If TestToHex_Works fails due to errors in ToHex, all tests relying on ToHex will end with different results and fail (ideally). The key here is detecting those different results. We do it making unit tests to be deterministic. As you do here

var bytes = new byte[] { 0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef };
Assert.AreEqual("0123456789abcdef", bytes.ToHex());

Further unit tests relying on ToHex should also be deterministic. If everything in ToHex goes well, the result should be the expected one. If you get a different one, something went wrong, somewhere and this is what you want from a unit test, to detect these subtle changes and fail fast.

1

It's a bit tricky with such a minimal example, but let's reconsider what we're doing:

Let's assume I wrote an extension method in c# for byte arrays which encodes them into hex strings, as follows:

Why did you write an extension method to do this? Unless you're writing a library to encode byte arrays into strings, this isn't likely to be the requirement you're trying to fulfill. What it looks like you were doing here was trying to fulfill the requirement "Validate some payload which is a byte array" - so the unit tests you're implementing should be "Given a valid payload X, my method returns true" and "Given an invalid payload Y, my method returns false".

So when you initially implement this, you might do the "byte-to-hex" stuff inline in the method. And that's fine. Then later you get some other requirement (e.g. "Display the payload as a hex string") which, while you're implementing it, you realise also requires you to convert a byte array to a hex string. At that point you create your extension method, refactor the old method, and call it from your new code. Your tests for the validation functionality shouldn't change. Your tests for displaying the payload should be the same whether the bytes-to-hex code was inline or in a static method. The static method is an implementation detail you shouldn't care about when writing the tests. The code in the static method will be tested by its consumers. I wouldn't even bother writing tests for it.

  • "this isn't likely to be the requirement you're trying to fulfill. What it looks like you were doing here was trying to fulfill the requirement " - you're making a lot of assumptions about the purpose of a function that most likely was just picked as a representative example, and not something out of a larger program. – whatsisname Jun 27 at 23:12
  • The "unit" in "unit testing" doesn't mean a single function, or class. It means a unit of functionality. Unless you're specifically writing a library that converted bytes to strings, that part is very much an implementation detail of some more useful bit of behaviour. That useful behaviour is the part you want to have tests for, because that's the bit you care about being correct. In an ideal world you want to be able to find a library already exists for doing the bin-to-hex (or whatever it is, don't get hung up on the specifics), swap it out for your implementation and not change any tests. – Chris Cooper Jun 28 at 8:22
  • The devil is in the details. Unit testing "implementation details" is still an enormously useful thing to do and is not something to skim over. – whatsisname Jun 28 at 16:58
0

Exactly. And this is one of the problems with static methods, another one being that OOP is a much better alternative in most situations.

This is also one of the reasons Dependency Injection is used.

In your case, you may prefer having a concrete converter that you inject into a class which needs a conversion of some values to hexadecimal. An interface, implemented by this concrete class, would define the contract required to convert the values.

Not only would you be able to test your code easier, but you'll also be able to swap implementations later (since there are lots of other possible ways of converting values to hexadecimal, some of them producing different outputs).

  • 2
    How will that not make the code dependant on the success of the converter tests ? Injecting a class, an interface or anything you want does not make it magically work. If you were talking about side effects then yes there would be a ton of value injecting a fake instead of hard coding an implementation, but that problem never existed in the first place anyway. – Steve Chamaillard Jun 26 at 16:19
  • 1
    @ArseniMourzenko so when injecting a calculator class implementing a multiply method, you'd mock this? Which means in order to refactor (and use the * operator instead), you'll have to change every single mock declaration in your code? Doesn't look like easy code to me. All I'm saying is toHex is very simple and has no side effects at all so I don't see any reason to mock or stub this. It's too much cost for absolutely no profit. Not saying we shouldn't inject it though, but that depends on usage. – Steve Chamaillard Jun 26 at 17:28
  • 2
    @Ewan, if I modified a piece of code and then faced "a sea of red", I could think oh no, where do I start?. But maybe I could just go back to the piece of code I just modified and look at its tests first to see what I broke. :p – David Arno Jun 27 at 7:29
  • 3
    @ArseniMourzenko, if DI makes your design so much better, why don't I see arguments for injecting types like String and int as well? Or basic utility classes from the standard library? – Bart van Ingen Schenau Jun 27 at 9:10
  • 4
    Your answer completely ignores practical aspects. There are plenty of situations with static methods that are fast, self-contained, and unlikely enough to ever change, that going to the effort of injecting them or jumping through any hoop other than a regular function call, is a pointless waste of time and effort. – whatsisname Jun 27 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.