Unit Testing a method that contains two output type parameters

Question

I was following a book Art of Unit Testing, This book says that You must not have any logic in Your asserts as this reduces readability of test, or I may be testing too many things at a time. Suppose there is a method with signature as below:

public int DoSomething(int variable1, out string variable2, out float variable3)
{
...
}

How should I test this method, as I have to assert on return value, variable2, variable3 ???

---

Sample Test method that tests all of these three concerns:

    public void DosSomethingTest(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        //this is where I am testing 3 concerns in one test. Is there a better way? 
        if(actualValue==expectedvalue){
            if(expectedVariable1==actualVariable1){
                if(expectedVariable3!=actualVariable3){
                    Assert.Fail("Unexpected variable2 returned");//pseudo code
                }
            }
            else{
                Assert.Fail("Unexpected variable1 returned");//pseudo code
            }
        }
        else{
            Assert.Fail("Unexpected Value returned");//pseudo code
        }
    }

Approach 2

---

Seperate Test methods that tests concerns separately:

    public void DosSomething_Passing15_Returns12Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualValue, expectedvalue);
    }

    public void DosSomething_Passing15_ReturnsExpectedVariable1Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualVariable1, expectedVariable1);
    }

    public void DosSomething_Passing15_ReturnsExpectedVariable2Test(){
        //add code for arrange
        int expectedvalue=12;
        string expectedVariable1="I am expected";
        string expectedVariable3=15.9;
        string actualVariable1, actualVariable3;
        var sut=new Foo();
        int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

        Assert.IsTrue(actualVariable3, expectedVariable3);
    }

Isn't approach 2 adhering to "Tests should not test more than one concern"?

Conclusion

---

If redesigning of SUT is possible:

1. There should ideally be no out parameter instead an aggregate class should be evaluated in Asserts.
2. If there are unrelated out parameters then SUT method should be refactored into multiple methods, then those new methods should have their own Test methods and they will then test only one concern.

If redesigning of SUT is not possible then as per the answer there will be three Asserts

Answer 1

If someone tells you there is something wrong with:

int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable2);

AssertEquals("actualValue", 1, actualValue);
AssertEquals("actualVariable1", 23, actualVariable1);
AssertEquals("actualVariable2", 14, actualVariable2);

then they are very likely being overly pedantic, applying a rule blindly without understanding anything of context. I mean, you probably generally should consider simplifying a function with multiple out arguments.

But if you have them, that's how you test them.

Answer 2

You call this function a number of times with chosen values for variable1. For each input value, you check that each of the output variables and return value match the expected result. You write those as single asserts on each expected value, so that a failure pinpoints the fault. Each input value and set of output values constitutes a single test.

The values you choose should span the range of permitted values, and should particularly explore any known values that represent edge conditions for the algorithm the function uses. Some people call this "grey box" testing (not quite black).

You also test (or attempt to test) the function for aberrant behaviour (perhaps variable1 must be positive, so you check that it fails correctly for negative values.).

A data driven approach works well, where all the relevant values are pre-computed and stored in a database or other structure. The actual unit test should contain little or no computation.

Answer 3

Words Of Wisdom for those brave souls who are embracing unit testing

After struggling with this question, and googling endless days, I finally came to different conclusion:

A good Unit Test should test only one concern, but it can test multiple aspects.

In my question, I am asking about multiple aspects, so it is Ok to have three asserts and may/ may not be neccessary to refactor the SUT(System under Test) method as per the given answers:

In cases where there are different concerns which essentially means:

Different inputs and their corresponding outputs

as shown below:

//Works Only in case of Nunit.

 [TestCase(15, "I am expected", 15.9, 12)]
 [TestCase(17, "I am expected1", 100, 12)]
 [TestCase(18, "I am expected2", 112, 12)]
 public void DoSomething_ReturnsExpectedTest(int inputValue, string expectedVariable1, int expectedVariable3, int expectedValue){
    //add code for arrange
    string actualVariable1, actualVariable3;
    var sut=new Foo();
    int actualValue=sut.DoSomething(15, out actualVariable1, out actualVariable3);

    Assert.IsTrue(actualValue, expectedvalue);
    Assert.IsTrue(actualVariable1, expectedVariable1);
    Assert.IsTrue(actualVariable3, expectedVariable3);    }

Answer 4

The idea behind "a method should test one thing" is so you can immediately see what is causing the problem when the test fails.

If you have an assert on three different things then you will have to see what one is going wrong exactly.

Another benefit of splitting them up is so you have an overview over what works and what doesn't when an assert fails. If you put all three in the same method and the first assert fails, you won't know if the latter two work. This will increase the difficulty for solving the problem: you won't know when you accidentally break another output parameter.

At this point you have to make a decision for yourself: do you consider these three output variables closely related or are they worth splitting up?

If they are closely related you can keep them together but if they aren't then I would advise you to put them in separate methods. You should extract their common logic to another method so you don't have to write the setup twice.

Answer 5

An assert is an encapsulation of the test logic:

assert foo(x) is y

can be written as

if foo(x) is not y then throw AssertionError

Using assert is encouraged since it's shorter, more readable, and unit testing frameworks may recognize it and treat it specially.

Now, you want to have those benefits for assertions that encapsulate a different logic - one that also tests out parameters. For this - you should write your own assertion helper!

The logic will be inside the assertion helper and the assertion statement itself will remain logicless. True - it's syntax will be a bit complex, as you have to specify the method to be tested, the input parameters, the expected return value and the expected out parameter values. Still, the fact that you have out parameters means you are using a language that is not Java so you should be able to achieve a nice syntax, that can be readable with the right line-breaking and indentation conventions.

Maybe your testing framework even have such helper!

UPDATE

You are concerned about testing multiple things with the assertion. OOP can help you here - first create an object that encapsulates the specific call to the method, and then use methods of that object to assert the different parts of the result of that call:

ComplexAssert.AssertWithOutParameters(sut.DoSomething,15)
    .AssertReturnValue(expectedValue,"Unexpected Value returned")
    .AssertOutParameter(1,expectedVariable1,"Unexpected variable1 returned")
    .AssertOutParameter(2,expectedVariable3,"Unexpected variable2 returned");

Now everything is neat and readable - the tested method is only called once and still each assertions gets it's own method call.