How does unit testing work?

Question

I am trying to make my code more robust and I have been reading about unit testing, but I find it very hard to find an actual useful use. For instance, the Wikipedia example:

public class TestAdder {
    public void testSum() {
        Adder adder = new AdderImpl();
        assert(adder.add(1, 1) == 2);
        assert(adder.add(1, 2) == 3);
        assert(adder.add(2, 2) == 4);
        assert(adder.add(0, 0) == 0);
        assert(adder.add(-1, -2) == -3);
        assert(adder.add(-1, 1) == 0);
        assert(adder.add(1234, 988) == 2222);
    }
}

I feel that this test is totally useless, because you are required to manually compute the wanted result and test it, I feel like a better unit test here would be

assert(adder.add(a, b) == (a+b));

but then this is just coding the function itself in the test. Can someone provide me with an example where unit testing is actually useful? FYI I am currently coding mostly "procedural" functions that take ~10 booleans and a few ints and give me an int result based on this, I feel like the only unit testing I could do would be to simply re-code the algorithm in the test. edit: I should also have precised this is while porting (possibly badly designed) ruby code (that I didn't make)

Answer 1

Unit tests, if you're testing small enough units, are always asserting the blindingly obvious.

The reason that add(x, y) even gets mention of a unit test, is because sometime later somebody will go into add and put special tax logic handling code not realizing that add is used everywhere.

Unit tests are very much about the associative principle: if A does B, and B does C, then A does C. "A does C" is a higher-level test. For instance, consider the following, completely legitimate business code:

public void LoginUser (string username, string password) {
    var user = db.FetchUser (username);

    if (user.Password != password)
        throw new Exception ("invalid password");

    var roles = db.FetchRoles (user);

    if (! roles.Contains ("member"))
        throw new Exception ("not a member");

    Session["user"] = user;
}

At first glance this looks like an awesome method to unit test, because it has a very clear purpose. However, it does about 5 different things. Each thing has a valid and invalid case, and will make a huge permutation of unit tests. Ideally this is broken down further:

public void LoginUser (string username, string password) {

    var user = _userRepo.FetchValidUser (username, password);

    _rolesRepo.CheckUserForRole (user, "member");

    _localStorage.StoreValue ("user", user);
}

Now we're down to units. One unit test does not care what _userRepo considers valid behavior for FetchValidUser, only that it's called. You can use another test to ensure exactly what a valid user constitutes. Similarly for CheckUserForRole ... you've decoupled your test from knowing what the Role structure looks like. You've also decoupled your entire program from being tied strictly to Session. I imagine all the missing pieces here would look like:

class UserRepository : IUserRepository
{
    public User FetchValidUser (string username, string password)
    {
        var user = db.FetchUser (username);

        if (user.Password != password)
            throw new Exception ("invalid password");

        return user;
    }
}

class RoleRepository : IRoleRepository
{
    public void CheckUserForRole (User user, string role)
    {
        var roles = db.FetchRoles (user);

        if (! roles.Contains (role))
            throw new Exception ("not a member");
    }
}

class SessionStorage : ILocalStorage
{
    public void StoreValue (string key, object value)
    {
        Session[key] = value;
    }
}

By refactoring you have accomplished several things at once. The program is way more supportive of tearing out underlying structures (you can ditch the database layer for NoSQL), or seamlessly adding locking once you realize Session isn't thread-safe or whatever. You've also now given yourself very straightfoward tests to write for these three dependencies.

Hope this helps :)

Answer 2

I am currently coding mostly "procedural" functions that take ~10 booleans and a few ints and give me an int result based on this, I feel like the only unit testing I could do would be to simply re-code the algorithm in the test

I am pretty sure that each one of your procedural functions is deterministic, so it returns a specific int result for every given set of input values. Ideally, you would have a functional specification from which you can figure out what result you should receive for certain sets of input values. In lack of that, you can run the ruby code (which is presumed to work correctly) for certain sets of input values, and record the results. Then, you need to HARD CODE the results into your test. The test is supposed to be a proof that your code does indeed produce results which are known to be correct.

Answer 3

I feel like the only unit testing I could do would be to simply re-code the algorithm in the test

You are nearly correct for such a simple class.

Try it for a more complex calculator.. Like a bowling score calculator.

The value of unit tests is more easily seen when you have more complex "business" rules with different scenarios to test for.

I'm not saying you shouldn't test a run of the mill calculator (Does your calculator account issues in values like 1/3 that can't be represented? What does it do with division by zero?) but you will see the value more clearly if you test something with more branches to get coverage on.

Answer 4

Since nobody else seems to have provided an actual example:

    public void testRoman() {
        RomanNumeral numeral = new RomanNumeral();
        assert( numeral.toRoman(1) == "I" )
        assert( numeral.toRoman(4) == "IV" )
        assert( numeral.toRoman(5) == "V" )
        assert( numeral.toRoman(9) == "IX" )
        assert( numeral.toRoman(10) == "X" )
    }
    public void testSqrt() {
        assert( sqrt(4) == 2 )
        assert( sqrt(9) == 3 )
    }

You say:

I feel that this test is totally useless, because you are required to manually compute the wanted result and test it

But the point is that you are much less likely to make a mistake (or at least more likely to notice your mistakes) when doing the manual computations then when coding.

How likely are you to make a mistake in your decimal to roman conversion code? Pretty likely. How likely are you to make a mistake when converting decimal to roman numerals by hand? Not very likely. That's why we test against manual calculations.

How likely are you to make a mistake when implement a square root function? Pretty likely. How likely are you make a mistake when calculating a square root by hand? Probably more likely. But with sqrt, you can use a calculator to get the answers.

FYI I am currently coding mostly "procedural" functions that take ~10 booleans and a few ints and give me an int result based on this, I feel like the only unit testing I could do would be to simply re-code the algorithm in the test

So I'm going to speculate about what's happening here. Your functions are kinda complicated, so it is hard to figure out from the inputs what the output should be. In order to do that, you have to manually execute (in your head) the function to figure out what the output is. Understandably, that seems kinda useless and error-prone.

The key is that you want to find the correct outputs. But you have to test those outputs against something known to be correct. Its no good to write your own algorithm to calculate that because that may very well be incorrect. In this case its too hard to manually calculate the values.

I'd go back to the ruby code, and execute these original functions with various parameters. I'd take the results of the ruby code and put those in the unit test. That way you don't have to do the manual calculation. But you are testing against the original code. That should help keep the results the same, but if there are bugs in the original then it won't help you. Basically, you can treat the original code like the calculator in the sqrt example.

If you showed the actual code you are porting we could provide more detailed feedback on how to approach the problem.

Answer 5

Despite religous zealotry about 100% code coverage, I will say that not every method should be unit tested. Only functionality that contains meaningful business logic. A function that simply adds number is pointless to test.

I am currently coding mostly "procedural" functions that take ~10 booleans and a few ints and give me an int result based on this

There is your real problem right there. If unit testing seems unnaturally hard or pointless then it is likely because of a design flaw. If it were more object oriented your method signatures would not be so massive and there would be fewer possible inputs to test.

I don't need to go into my OO is superior to procedural programming...

Answer 6

In my point of view unit tests are even usefull at your little adder class: don't think of "recoding" the algorithm and think of it as a black box with the only knowledge you have about is the functional behaviour (if you are familiar with fast multiplication you know some faster, but complexer attempts than using the "a*b") and your public interface. Than you should ask yourself "What the hell could go wrong?"...

In the most cases it happens at the border (i see you test already adding these patterns ++, --, +-, 00 - time to complete these by -+, 0+, 0-, +0, -0). Think of what happens at MAX_INT and MIN_INT when adding or subtracting (adding negatives ;) ) there. Or try to make sure your tests look quite exactly what happens at and around zero.

All in all the secret is very simple (maybe for complexer ones too ;) ) for simple classes: think about the contracts of your class (see design by contract) and then test against them. The better you know your inv's, pre's and post's the "completer" your tests will be.

Hint for your test classes: try to write only one assert in a method. Give the methods good names (e.g. "testAddingToMaxInt", "testAddingTwoNegatives") to have the best feedback when your test fails after code change.

Answer 7

Rather than testing for a manually calculated return value, or duplicating the logic in the test to calculate the expected return value, test the return value for an expected property.

For example, if you want to test a method that inverts a matrix, you don't want to manually invert your input value, you should multiply the return value by the input and check that you get the identity matrix.

To apply this approach to your method, you will have to consider its purpose and semantics, to identify what properties the return value will have relative to the inputs.

Answer 8

Unit tests are a productivity tool. You receive a change request, implement it, then run your code through the unit test gambit. This automated testing saves time.

I feel that this test is totally useless, because you are required to manually compute the wanted result and test it, I feel like a better unit test here would be

A moot point. The test in the example just shows how to instantiate a class and run it through a series of tests. Concentrating on the minutia of a single implementation is missing the forest for the trees.

Can someone provide me with an example where unit testing is actually useful?

You have an Employee entity. The entity contains a name and address. The client decides to add a ReportsTo field.

void TestBusinessLayer()
{
   int employeeID = 1234
   Employee employee = Employee.GetEmployee(employeeID)
   BusinessLayer bl = new BusinessLayer()
   Assert.isTrue(bl.Add(employee))//assume Add returns true on pass
}

That's a basic test of the BL for working with an employee. The code will pass/fail the schema change you just made. Remember that assertions are not the only thing the test does. Running through the code also ensures no exceptions are bubbled up.

Over time, having the tests in place makes it easier to make changes in general. The code is automatically tested for exceptions and against Assertions you make. This avoids much of the overhead incurred by manual testing by a QA group. While UI is still pretty difficult to automate, the other layers are generally very easy assuming you use access modifiers correctly.

I feel like the only unit testing I could do would be to simply re-code the algorithm in the test.

Even procedural logic is easily encapsulated inside a function. Encapsulate, instantiate and pass in the int/primitive to be tested (or mock object). Do not copy paste the code into a Unit Test. That defeats DRY. It also defeats the test entirely because you are not testing the code, but a copy of the code. If the code that should have been tested changes, the test still passes!

Answer 9

Taking your example (with a bit of refactoring),

assert(a + b, math.add(a, b));

doesn't help to:

• understand how math.add behaves internally,
• know what will happen with edge cases.

It's pretty much as saying:

• If you want to know what the method does, go and see the hundreds of lines of source code yourself (because, yes, math.add can contain hundreds of LOC; see below).
• I don't bother to know if the method works correctly. It's ok if both expected and actual values are different from what I really expected.

This also means that you don't have to add tests like:

assert(3, math.add(1, 2));
assert(4, math.add(2, 2));

They don't help neither, or at least, once you made the first assertion, the second one brings nothing useful.

Instead, what about:

const numeric Pi = 3.1415926535897932384626433832795;
const numeric Expected = 4.1415926535897932384626433832795;
assert(Expected, math.add(Pi, 1),
    "Adding an integer to a long numeric doesn't give a long numeric result.");
assert(Expected, math.add(1, Pi),
    "Adding a long numeric to an integer doesn't give a long numeric result.");

This is self-explanatory and damn helpful both for you and for the person who will maintain the source code later. Imagine that this person does a slight modification to the math.add to simplify the code and optimize the performance, and sees the test result like:

Test TestNumeric() failed on assertion 2, line 5: Adding a long numeric to an
integer doesn't give a long numeric result.

Expected value: 4.1415926535897932384626433832795
Actual value: 4

this person will understand immediately that the newly modified method depends on the order of the arguments: if the first argument is an integer and the second one is a long numeric, the result would be an integer, while a long numeric was expected.

In the same way, obtaining the actual value of 4.141592 at the first assertion is self-explanatory: you know that the method is expected to deal with a big precision, but actually, it fails.

For the very same reason, two following assertions can make sense in some languages:

// We don't expect a concatenation. `math` library is not intended for this.
assert(0, math.add("Hello", "World"));

// We expect the method to convert every string as if it was a decimal.
assert(5, math.add("0x2F", 5));

Also, what about:

assert(numeric.Infinity, math.add(numeric.Infinity, 1));

Self-explanatory too: you want your method to be able to deal correctly with the infinity. Going beyond infinity or throwing an exception is not an expected behavior.

Or maybe, depending on your language, this will make more sense?

/**
 * Ensures that when adding numbers which exceed the maximum value, the method
 * fails with OverflowException, instead of restarting at numeric.Minimum + 1.
 */
TestOverflow()
{
    UnitTest.ExpectException(ofType(OverflowException));

    numeric result = math.add(numeric.Maximum, 1));

    UnitTest.Fail("The tested code succeeded, while an OverflowException was
        expected.");
}

Answer 10

For a very simple function like add, testing could be considered unnecessary, but as your functions become more complex, it becomes more and more obvious why testing is necessary.

Think about what you do when you're programming (without unit testing). Usually you write some code, run it, see that it works, and move onto the next thing, right? As you write more code, especially in a very large system/GUI/website you find that you have to do more and more "running and seeing if it works". You have to try this and try that. Then, you make a few changes and you have to try those same things all over again. It becomes very obvious that you could save time by writing unit tests that would automate the whole "running and seeing if it works" part.

As your projects become larger and larger the number of things you have to "run and see if it works" for becomes unrealistic. So you end up just running and trying a few major components of the GUI/project and then hoping everything else is fine. This is a recipe for disaster. Of course you, as a human being, can't repeatedly test for every single possible situation that your clients might use if the GUI is used by literally hundreds of people. If you had unit tests in place, you could simply run the test before shipping out the stable version, or even before committing to the central repository (if your workplace uses one). And, if there are any bugs found later on, you can just add a unit test to check for it in the future.

Answer 11

One of the benefits of writing unit tests is that it helps you to write more robust code by forcing you to think about edge cases. How about testing for some edge cases, like integer overflow, decimal truncation, or handling of nulls for the parameters?

Answer 12

Maybe you assume add() was implemented with the ADD instruction. If some junior programmer or hardware engineer reimplemented the add() function using ANDS/ORS/XORS, bit inverts and shifts, you might want to unit test it against the ADD instruction.

In general, if you replace the guts of add(), or the unit under test, with a random number or output generator, how would you know that something was broken? Encode that knowledge in your unit tests. If nobody can tell if it's broken, then just check in some code for rand() and go home, your job is done.

Answer 13

I might have missed it in amongst all the replies but, to me, the main drive behind Unit Testing is less about proving the correctness of a method today but that it proves the continued correctness of that method when[ever] you change it.

Take a simple function, like returning the number of items in some collection. Today, when your list is based on one internal data structure that you know well, you might think that this method is so painfully obvious that you don't need a test for it. Then, in several months or years time, you (or someone else) decide[s] to replace the internal list structure. You still need to know that getCount() returns the correct value.

That's where your Unit Tests really come into their own.

You can change the internal implementation of your code but to any consumers of that code, the result remains the same.