When to use asserts vs unit tests or both?

Question

Background:

I'm creating my first C++ game project and I plan on calling a InitialThirdPartySystems() function to...well...intialize all third party systems to be used in my code lol. Within it I want to do some checks to make sure all systems are initializing properly. However, I'm not quite sure of the properly way to have these checks while also creating my unit tests for this function. For example, I would normally assert that all these systems initialize properly since my game will depend on these systems functioning properly:

void InitializeThirdPartySystems()
{
   assert(Init_System(), "Error: System not initialized!");
}

So this would work fine within my debug build, but since assertions will be turned off in my release build then this check will go away. To solve this, I could design the function to be tested like so:

bool InitializeThirdPartySystems()
{
   if (!Init_System())
   {
      LOG("System Failed!");
      return false
   }
   return true;
}

So now I can just have my unit test check to see if systems are initializing properly or not. However, I won't be running any automatic tests in the background for now so that means I would have to keep switching to my test project to build and test my code.

The third option would be to do both kinds of testing (asserts and if style checks for unit tests) which would cover all my bases and would be good so that I always have asserts double checking anything my tests might miss. However, this will bloat my code and I generally haven't seen things implemented this way by others.

Question:

What is typically the best way to go about asserting/testing your code? Are my options the only ones or is there other ways to go about this I'm missing? Thanks for any input.

Answer 1

Neither asserts nor unit tests are appropriate in that case. Both are meant to check that your code actually does what you want it to do. This is not part of your application, and only run at development/debugging time.

Unit tests check some small chunk of code does what is needed, provided other components are correct.

Asserts document that you designed your code so that some condition is always (meant to) be true at that time, taking into account the interactions between components.

A third party system failing to initialize is out of your direct control.

You just have to detect it (check a status ?, catch exception ?,...) and handle it (intelligently abort the program with appropriate information, propose some degraded mode, ...). This is part of your application release code and runs in production.

Answer 2

assert() is used to abort() the program execution when the precondition fails.

In general, simply aborting a program is an undesirable outcome - whether in a safety critical application, or a game (as in your case).

But let's assume that you will not be shipping with the assert() still compiled in... you have then removed that safety net.

While unit testing may eliminate most (if not all) sources of errors, you would be better preventing the erroneous condition (eg by bounds checking and do something with it).

Answer 3

assert is a low-level mechanism that could be used (among other things) to implement your unit tests. It can also be useful for asserting preconditions and/or postconditions on your code.

Unit tests typically come in two varieties: those that should log an error, then continue testing, and those that immediately abort if a test fails. Clearly an assert can only be used for the latter variety.

The main strength (and weakness) of assert is that it immediately aborts the program, with no attempt at remediation, clean shutdown, or anything else. It's the "do not pass Go, do not collect $200" option.

This is useful when you've detected a condition so problematic that attempting to shut down cleanly is more likely to lead to more problems, or possibly destroy the user's data. Just for one obvious case, you've detected that the stack has been corrupted, so even attempting to return from the current function may start executing invalid code. Likewise, if you detect that the heap has been corrupted (especially in a program that uses the heap heavily) your safest course of action may be to shut down immediately, to at least prevent further damage to the system. As @Andrew said, this isn't a desirable outcome--but if the system has sufficient problems, it may be the least undesirable option available.

The other major characteristic of assert is what you've noticed: in a release build it'll normally be compiled out so the expression in the assert isn't evaluated¹.

This indicates that if you use assert directly in your code (as opposed to implementing a unit test, or similar) it should be reserved for cases where the logic remains complete even when it's absent. This can be things like testing that the stack and heap haven't bee corrupted. It can also be simply assuring the consistency of the programs internal logic. It definitely means that an assert should not normally depend on input to the program, or anything along that line.

In the case you cited, you clearly can't put initialization into the assert itself. It might make sense to use an assert on the return value (but more likely not). The question at that point is whether you can meaningfully continue execution at all. If failed initialization means you need to shut down immediately, no matter what, then an assert can make sense. If you can continue to the point of logging the result and shutting down more cleanly, then an assert isn't appropriate. If you might even be able to continue execution, and just have some features disabled, then an assert would clearly be entirely the wrong tool for the job.

Even in a case where you can't continue execution, using assert to test the return value is probably somewhat questionable. The initialization is (apparently) designed so it may return false in case of failure. If it does so, that's still correct operation. The more fitting use of assert would be to test its post-conditions in case of success. For example, let's assume that the initialization is required to set up a global pointer to some program parameters (and we'll leave discussing the wisdom of that design aside for the moment), in case of success it might make sense to assert that the pointer in question is non-null (because if the initialization succeeded and worked correctly, it must always be non-null).

In his comment, David Hammen pointed to some languages that omit assert-like mechanisms on the grounds that: "the authors of those languages think assert is evil, perhaps more so than is goto." While I don't particularly doubt that at least one such ignorant buffoon has designed (and perhaps even implemented) a language, any such language should be avoided. That's not to say that assert is particularly necessary--only that anybody who so clearly misunderstands something so simple shouldn't be trusted to tie his own shoes, not to mention designing a programming language.

If you wanted to question something (and you had a brain) what you'd question would be the inclusion of a fairly specialized construct in the standard library, when others that are likely to be of much wider use are omitted--especially in a library like C's that's missing so much in the way of fundamental functionality that most programs need far more often than they need assert. This position would be reinforced by the fact that assert is quite trivial to implement on your own (a half dozen lines or so of simple code) while common tasks that involved a great deal more boiler plate are left to users to implement and re-implement in nearly every program they write.

That's a valid question, but (I think) it does have an answer (and a pretty simple one). When the C89/90 standard was written, a fair number of people were still doing development on small machines with extremely limited RAM and mass storage (e.g., a single floppy disc). assert was something they could add that was fairly widely usable, and still small enough that it wouldn't cause a major problem for such limited systems.

It would also be possible (entirely reasonable, IMO) for somebody to point out that macros, as defined and implemented in the C preprocessor are seriously problematic--and as it happens, assert is required to be a C preprocessor macro. If you eliminate preprocessor macros, you eliminate assert (as it's defined in C today). I would assert (pardon the semi-pun) that this is a reasonable approach--in fact, it's probably fair to say that there's only one good reason to design a new language that includes the C preprocessor--to maintain compatibility with existing C source code (e.g., new versions of C or C++). For just about anybody else, using the C preprocessor (or its macros) would be highly questionable at best.

---

^{1. Interestingly, however, it still has to compile to a syntactically valid expression (typically something like (void)0). Although almost nobody really does, it's officially valid to use assert in places that expanding to an empty string would produce invalid syntax (e.g., something ? assert(x) : assert(y);).}