2

It seems the general consensus for unit testing classes is to test your object through its public interface only. So if you wanted to test the removeElement method on a LinkedList class you'd need to call addElement, then removeElement, and lastly containsElement to assert the element was removed.

This is brittle because if addElement or containsElement broke, then the test will fail even if the implementation of removeElement is correct.

When I test standalone procedures I try to call them in isolation. If I were to test a removeElement procedure I would build up the state of the parameters directly in the test and then assert their state post call is correct.

The only difference between a method and a procedure is that a method is implicitly given the object as a parameter. So since list.removeElement(el) and removeElement(list, el) are functionally the same, why not test them the same way? e.g. In the test, create an instance of the LinkedList class, setup up its "private" fields, call removeElement, and assert its fields post-call changed correctly.

This is the ideal unit test because its about taking input and asserting output for a single unit of functionality. Having to call public methods A, B, C, D, E, and F just to test method G is a borderline integration test, can potentially create false positives (since the data itself is never validated), and makes isolating the failure of the test during maintenance more difficult.

Anecdotally I've found that black box testing tempts developers to add unnecessary public methods to make their testing "easier" but increases maintenance in the long run.

So my question is why is white box testing discouraged in the OO world when it seems like common sense in the procedural and functional worlds?

EDIT: Is there an OO way of dealing with the grips I've outlined in my post, specifically in the later half that do not involve adding new public methods and avoid calling public methods other than the one being tested? Consider the dilemma of asserting the "previous" node pointer in my comment.

EDIT #2: Apparently my concept of a class might be different from others. A class to me is just a (very old) design pattern: "construct", "consume" (e.g. call methods), and "destruct" which is no different than fopen, fread, fwrite fseek, and fclose in C. Regardless of whether there is an implicit parameter involved, things are stuffed behind a namespace, or you call it private, public, or protected everything is just data and data transformations at the end of the day. I'm having trouble grasping classes as a unit when it seems more like a design pattern or even "container" for the actual units which are the functions themselves.

  • 6
    "Anecdotally I've found that black box testing tempts developers to add unnecessary public methods to make their testing "easier" ...." Isn't this exactly what you are suggesting by "setting up private fields"? – Euphoric Jun 18 '17 at 18:12
  • And I don't think accessing "encapsulated" fields in structures is common sense in "procedural" style and it is not a problem with immutable structures in functional style. – Euphoric Jun 18 '17 at 18:19
  • 5
    "You should be able to test functions one-at-at-time". I'd actually call that a (widely spread) antipattern that is impossible to achieve anyway, because in each test you will at least call a constructor. Also it generally doesn't make sense to test functions that are supposed to work with each other only in isolation (think e.g. about push and pop). Unit tests are not a replacement for a debugger. – MikeMB Jun 18 '17 at 19:18
  • 3
    I think this boils down to question of encapsulation. You seem to believe that it should be fine for internals of class to be exposed to test, which I highly disagree with. As it would make refactoring harder and would require future maintainers of the tests to know not only API of the class, but also it's internals. – Euphoric Jun 18 '17 at 19:32
  • 1
    I don't consider them gripes. And even if I would, they would be less worrisome than leaking privates outside of class. Also, I don't consider it is possible to isolate method itself. Unit test in OOP is a class, not a method. And single unit test tests single behavior of class, not single method. And behavior of class might require calls of multiple methods. – Euphoric Jun 18 '17 at 20:02
14

You are mixing up two related, but nevertheless different things:

  • white box testing

  • unit testing by using private methods

The reasons for writing or not writing unit test only using public methods have been discussed numerous times before on this site, for example here or here. I don't think it makes sense to repeat those arguments, if that is your question, you will probably find an answer following those links.

White box testing, however, does not mean to use private members for setting up a test. It means to design tests using specific knowledge about the internals of the tested class or component. For example, by creating tests to achieve full code coverage and/or branch coverage - and this is typically done by using just public members. So white box testing requires to know the internals of a class, but does not directly utilize access to the internals. This lets the designer of a component in a situation where he can still change the implementation details without worrying too much about the tests.

This kind of testing is not discouraged in OOP, quite the opposite. The well known Test Driven Development (which is popular in OOP as well as in non-OOP) is a form of testing which actually leads to these kind of tests: whenever one wants to add a new feature to a function, class of component, one writes a "red" test first, adds some new code or changes some existing code to add the feature, and since the new test now becomes "green", it is obvious the added or changed code must have been covered by the test.

To your example: if removeElement is a public method of a "list" module, and not a member of a class, I would still recommend the way of testing using only the public interface of that module, just as if it was a class. Your example of a broken addElement or containsElement is contrived (and your idea of "to avoid calling public methods other than the one being tested" is - no offence - misguided). In reality, one would design such a test by

  • creating a new list
  • assert the list does not contain element X
  • add an element X to the list
  • assert the list nows contain element X
  • remove the element X from the list
  • assert the list does not contain element X any more

which is all possible using public methods.

If addElement or containsElement were broken, the above test sequence makes sure the test wil reveal this (and does not give a false positive for removeElement).

Of course, there are cases of classes or complex components where using the public interface alone might not be the best approach to create a full test scenario, and where it can be helpful to loosen the encapsulation to some degree, for example, by adding "maintenance hatches" into the code. But I think such cases are exceptional cases, and good test and component design should try to avoid these situations. A simple component like a linked list should not require such measures.

  • 1
    "one would design such a test by ... " Yes, but what Zippers is saying that such test would break if any of the public methods broke, not only if remove broke. Which is what seems to bothering him. – Euphoric Jun 18 '17 at 18:31
  • 2
    @Euphoric: to me it seems the OP is bothered more by the fact a test by using just add, remove and contains can result in a false positive, which can be mitigated by adding assert statements after each step. – Doc Brown Jun 18 '17 at 18:36
  • 6
    @Zippers "If you wanted to test removing the middle element from a doubly linked list how would you assert the "previous" pointer on the next node was correct after the removal?" Unittesting does not verify that. Unittesting verifies that the iteration skips the removed element or that it is inaccessible through the public interface because this is the expected behavior, That pointers are involved which have to be adjusted is state or implementation detail which is not vefified. This way the implementation detail can change (improve) without braking the test. – Timothy Truckle Jun 18 '17 at 21:12
  • 2
    The quoted Wikipedia definition seems to be wrong. White-box testing doesn't test internals etc. but uses knowledge of the implementation to determine inputs that are more likely to cause trouble than others. Best or worst case depending on your point of view would be me reading the source code, finding a bug, and writing a test that I know exhibits the bug and fails - a test that would have been impossible to create with black-box testing. – gnasher729 Jun 19 '17 at 11:20
  • 2
    @Zippers Well, it doesn't matter much if the internal pointer is set wrong if the actual traversal of the list provides the right order. Assume my double-linked list looks like A-B-C with an interface with add/remove/next/previous functions. If I can call remove(B) while at A, then call next() and be at C, then call previous() and be at A, then the implementation details don't matter. The thing works, and tests exist to know that behavior functions, not to verify that the code hasn't changed one bit (like if we change nextPtr and prevPtr to a tuple ptrs=(next,prev), your test would break). – Delioth Jun 19 '17 at 19:16
0

I think whats going on here is a difference of perspective: Do you view classes as the smallest unit or functions? I'm going to deconstruct my question from the perspective of someone who views classes as the smallest unit:

[The] ideal unit test ... is about taking input and asserting output for a single unit of functionality. Having to call public methods A, B, C, D, E, and F just to test method G is a borderline integration test.

This makes no sense because the class is the smallest unit, therefore calling multiple methods isn't an issue because it tests the unit as a whole.

[It] can potentially create false positives (since the data itself is never validated).

Its not about data confidence its about the interface and if it appears to behave correctly.

Isolating the failure of the test during maintenance is more difficult.

If a test fails then it doesn't matter which method induced the failure because the unit as a whole is broken. Difficulty finding the root cause is accepted because the result fixes the unit as a whole.

If you wanted to test the removeElement method on a LinkedList class you'd need to call addElement, then removeElement, and lastly containsElement to assert the element was removed. This is brittle because if addElement or containsElement broke, then the test will fail even if the implementation of removeElement is correct.

Its not about a method failing, its about the class as a whole failing. Testing a specific method in isolation doesn't make sense. Its best to test the class as a whole through workflow or requirement tests.

Now I'll state the perspective of someone viewing functions as the smallest unit. I think I described it best in edit #2 (I've modified the wording slightly):

A class is a design pattern: "construct", "consume", and "destruct" which is no different than fopen, fread/fwrite/fseek, and fclose in C. Regardless of whether there is an implicit self parameter, definitions are placed in a namespace, or you call it private, public, or protected everything is data and data transformations at the end of the day.

From this perspective there is nothing wrong with establishing data inputs directly during testing. If you don't then you end up playing games with the function call sequence to nudge the data (which you weren't supposed to know about) into a state which you can indirectly verify; this sacrifices direct data confidence and test clarity.

There seems to be a divide regarding whether engineers should think in terms of data and data transformations or abstractions and at what level during development and testing. This raises the question as to why this divide exists. Its an important question because it has a ripple effect on how engineers view, design, and test software. I'm tempted to post a separate question because I believe its worth asking.

  • fopen, fread etc. are operations on a file, using an opaque FILE pointer. You surely do not want to write any tests utilizing the knowledge about the internals of FILE. The file abstraction in C is not IMHO not different from a file abstraction by a separate file class, in C the smallest sensible testable unit for a file is the data type FILE together with its operations. So I don't see how writing unit tests for an equivalent File class (like the File classes in Java or C#) should be quite different from writing unit tests for file operations in C. – Doc Brown Jun 20 '17 at 12:01
  • ... and you will also need to use multiple of these methods in a test. If you want to test fread, you need to utilize fopen. For a meaningful test of fseek, you will probably utilize fread to verify you reached the intended file position. – Doc Brown Jun 20 '17 at 12:09
  • I appreciate your feedback @DocBrown but your getting hung up on semantics and examples and your not addressing the core ideas: Do you believe software is about data and data transformation? Do you believe classes are an implementation of the "life cycle" pattern? – Zippers Jun 20 '17 at 16:18
  • Don't get me wrong, but as long as your examples seem to proof the opposite of what you are describing else, as long as your usage of terms like "whitebox testing" looks to be based on a misunderstanding or misconception, it is hard for me to understand what your core question is about. Surely software is about "data and data transformation" from some point of view, on the right level of abstraction, and I have only a vague idea what you mean by "life cycle pattern". But it seems you are drifting away from the original question, and this is a Q&A site, not a discussion forum. – Doc Brown Jun 20 '17 at 18:44
  • @DocBrown The point I was making is that how engineers view software has a ripple effect on how they design, test, and recommend test strategies. I suspect the emphasis on behavioral testing is driven by business needs, cost, engineering time, and quality assurance. To businesses, behavioral testing may be "good enough", but different industries and applications may need different levels of validation and correctness. – Zippers Jun 20 '17 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.