Unit testing 'opaque' classes / structs

Question

I'm sorry, no good title.

I'm working on a C project right now and am writing tests. I usually do a 'struct method' style of programming. So I have opaque structs (no implementation revealed outside of the .c file) and methods that act on pointers to them. But this applies to OOP as well.

Typically, when I am unit testing I want tests to be short and self-contained. I don't strictly adhere to the one assert per test, but I try to keep them small. Usually I more strictly follow one test per method.

However, I'm running into an issue that has come up before. So, for instance, I have a stack with (struct_handle being my opaque pointer):

int size(struct_handle);
int push(struct_handle);
int pop(struct_handle);
int peek(struct_handle);

So, I'd like to have (at least) a test for each of these 4 methods, and more over have only one assert in each of these tests. But, since my handle is opaque, to set up something like "did it pop what I pushed?" I need to rely on the push function working correctly. This is my problem, since I can't (and don't want to) access the implementation details, my tests must assume that some of my methods work when testing others.

Here are the ideas I've come up with:

• Ensure a testing order on these methods. That is, ensure that push works before testing pop since we need to push before we pop.
  • Issues: There might not always be a good / clear ordering of the methods.
  • Sometimes the methods are interdependent. How can I know push worked if I can't pop?
  • On that, even if there is a bit of an intuitive ordering (I need to push and pop before I can check the size) it might not be so in the details. That is, in the implementation, push and pop both rely on size to check for errors! So testing it that way doesn't actually ensure that push and pop work well before using size.
• So, if we can't test each method individually, I can instead test overall behavior.
  • I have one big test method for 'the stack functions'. Maybe it goes like 'check size is 0, try to pop empty, push to stack, check size is 1, pop that from stack, check correct value returned...'
  • While this seems well and nice (and I believe it is), lacks the benefit you get from one assert (or as few as possible) per test. You might fix one thing but then there are more issues behind it that just didn't get evaluated.

Anyone had experience with an issue like this? I'm curious as to everyone's opinions. Happy to share more details / stronger examples if asked.

Answer 1

I need to rely on the push function working correctly. This is my problem, since I can't (and don't want to) access the implementation details, my tests must assume that some of my methods work when testing others.

That's fine. And normal.

Which is to say, your problem isn't that your opaque data structures are difficult to unit test; your problem is that the definition of "unit test" that you've internalized includes constraints that are actively inhibiting you. (That's not your fault - the literature around "unit tests" went straight into the trash in the late 1990s, and still hasn't recovered.)

---

For data structures, my heuristic is "reads before writes" -- write-only data structures aren't actually useful. So we focus first on creating the tests we need to verify that we can read data.

HANDLE h = create_handle();
assert 0 == size(h)

Then we can focus on ensuring that what we read changes as we expect when we call functions that should change the data structure

HANDLE h = create_handle();
push(h);
assert 1 == size(h)

HANDLE h = create_handle();
push(h);
push(h);
push(h);
push(h);
pop(h);
push(h);
pop(h);
pop(h);
assert 2 == size(h)

And so on.

While this seems well and nice (and I believe it is), lacks the benefit you get from one assert (or as few as possible) per test. You might fix one thing but then there are more issues behind it that just didn't get evaluated.

Yes. The way I think of this is that we want to have a "suite" of tests, where each test makes a single measurement, and then checks that the measurement satisfies some specification, but our suite contains as many tests as we need to establish that our implementation is ready to ship.

Of course, a suite of tests introduces its own concerns (how do you keep track of which tests are running, passing, failing? Does test execution stop on the first failure, or run the complete suite and report? How much "busy work" is required to add a new test to the suite?)

It's trade offs all the way down.

Answer 2

OK its been a while since I did any c, but this seems like a common problem to me. Let me see if i understand your problem.

1. All the fields of your struct are private and cant be accessed. You can only call the methods.

2. The methods, (or at least some of them) only change the internal state of the struct, so there is no measurable change from outside you can check to see if the function worked correctly

3. You can use a combination of methods to check the overall functionality, but you would like to have a unit test for each method by itself.

If it were c# I would say inject the storage layer on the class construction, so that in your test you can inject and hold a reference to the storage which is checkable in the test.

With the opaque struct pattern I understand its good practice to make fake constructors and destructors, a mystruct_create and mystruct_destroy? you could pass in whatever type you are using to store your stack of data

excuse my c

target = mystruct_create(mystack)
push(target, element)
assert(mystack[0] == element)

this gives you access to the internals while still keeping them private.