Should there be assertions in release builds

Question

The default behavior of assert in C++ is to do nothing in release builds. I presume this is done for performance reasons and maybe to prevent users from seeing nasty error messages.

However, I'd argue that those situations where an assert would have fired but was disabled are even more troublesome because the application will then probably crash in an even worse way down the line because some invariant was broken.

Additionally, the performance argument for me only counts when it is a measurable problem. Most asserts in my code aren't much more complex than

assert(ptr != nullptr);

which will have small impact on most code.

This leads me to the question: Should assertions (meaning the concept, not the specific implementation) be active in release builds? Why (not)?

Please note that this question is not about how to enable asserts in release builds (like #undef _NDEBUG or using a self defined assert implementation). Furthermore, it is not about enabling asserts in third party/standard library code but in code controlled by me.

Answer 1

The classic assert is a tool from the old standard C library, not from C++. It is still available in C++, at least for reasons of backwards compatibility.

I have no precise timeline of the C standard libs at hand, but I am pretty sure assert was available shortly after the time when K&R C came into live (around 1978). In classic C, for writing robust programs, adding NULL pointer tests and array bounds checking needs to be done way more frequently than in C++. The gross of NULL pointer tests can be avoided by using references and/or smart pointers instead of pointers, and by using std::vector, array bounds checking is often unnecessary. Moreover, the performance hit at 1980 was definitely much more important than today. So I think that is very likely the reason why "assert" was designed to be active only in debug builds by default.

Moreover, for real error handling in production code, a function which just tests some condition or invariant, and crashes the program if the condition is not fulfilled, is in most cases not flexible enough. For debugging that is probably ok, since the one who runs the program and observes the error has typically a debugger at hand to analyse what happens. For production code, however, a sensible solution needs to be a function or mechanism which

• tests some condition (and stops execution at the scope where the condition fails)

• provides a clear error message in case the condition does not hold

• allows the outer scope to take the error message and outputs it to a specific communication channel. This channel might be something like stderr, a standard log file, a message box in a GUI program, a general error handling callback, a network-enabled error channel, or whatever fits best to the particular piece of software.

• allows the outer scope on a per-case base to decide if the program should end gracefully, or if it should continue.

(Of course, there are also situations where ending the program immediately in case of an unfulfilled condition is the only sensible option, but in such cases, it should happen in a release build as well, not just in a debug build).

Since the classic assert does not provide this features, it is not a good fit for a release build, assumed the release build is what one deploys into production.

Now you can ask why there is no such function or mechanism in the C standard lib which provides this kind of flexibility. Actually, in C++, there is a standard mechanism which has all these features (and more), and you know it: it is called exceptions.

In C, however, it is hard to implement a good, general purpose standard mechanism for error handling with all the mentioned features because of the lack of exceptions as part of the programming language. So most C programs have their own error handling mechanisms with return codes, or "goto"s, or "long jumps", or a mixture of that. These are often pragmatic solutions which fit to the particular kind of program, but are not "general purpose enough" to fit into the C standard library.

Answer 2

If you find yourself wishing asserts were enabled in a release you've asked asserts to do the wrong job.

The point of asserts is that they aren't enabled in a release. This allows for testing of invariants during development with code that would otherwise have to be scaffolding code. Code that has to be removed before release.

If you have something that you feel should be tested even during release then write code that tests it. The If throw construct works very well. If you wish to say something different than the other throws simply use a descriptive exception that says what you wish to say.

It's not that you can't change how you use asserts. It's that doing so isn't getting you anything useful, runs counter to expectations, and leaves you with no way clean way to do what asserts were meant to do: add tests that are inactive in a release.

I am not talking about a specific implementation of assert but the concept of an assertion. I don't want to misuse assert or confuse readers. I meant to ask why it is this way in the first place. Why is there no additional release_assert? Is there no need for it? What is the rationale behind assert being disabled in release? - Nobody

Why no relase_assert? Frankly because asserts aren't good enough for production. Yes there's a need but nothing fulfills that need well. Oh sure you can design your own. Mechanically your throwIf function just needs a bool and an exception to throw. And that may fit your needs. But you're really limiting the design. That's why it doesn't surprise me that there isn't a baked in assert like exception throwing system in your languages library. It certainly isn't that you can't do it. Others have. But dealing with the case where things go wrong is 80% of work for most programs. And so far no one has shown us a good one size fits all solution. Dealing effectively with these cases can get complicated. If we'd had a canned release_assert system that fell short of our needs I think it would have done more harm then good. You're asking for a good abstraction that would mean you wouldn't have to think about this problem. I want one too but it doesn't look like we're there yet.

Why are asserts disabled in release? Asserts were created at the height of the age of scaffolding code. Code we had to remove because knew we didn't want it in production but knew we wanted to run in development to help us find bugs. Asserts were a cleaner alternative to the if (DEBUG) pattern that let us leave the code but disable it. This was before unit testing took off as the main way to separate testing code from production code. Asserts are still used today, even by expert unit testers, both to make expectations clear and to cover cases that they still do better than unit testing.

Why not just leave debugging code in production? Because production code needs to not embarrass the company, not format the hard drive, not corrupt the database, and not send the president threatening emails. In short it's nice to be able to write debugging code in a safe place where you don't have to worry about that.

Answer 3

Assertions are a debugging tool, not a defensive programming technique. If you want to perform validation under all circumstances, then perform validation by writing a conditional – or create your own macro to reduce the boilerplate.

Answer 4

If you don't want an "assert" to be turned off, feel free to write a simple function that has a similar effect:

void fail_if(bool b) {if(!b) std::abort();}

That is, assert is for tests where you do want them to go away in the shipped product. If you want that test to be part of the defined behavior of the program, assert is the wrong tool.

Answer 5

assert is a form of documentation, like comments. Like comments, you would not normally ship them to customers - they don't belong in release code.

But the problem with comments is that they can become outdated, and yet they're left in. That's why asserts are good - they're checked in debug mode. When the assert becomes obsolete, you quickly discover it, and will still know how to fix the assert. That comment which became outdated 3 years ago? Anybody's guess.

Answer 6

I'm coding in Ruby, not C/C++, and so I won't speak about difference between asserts and exceptions but I would like to talk about it as a thing that stops the runtime. I disagree with most of answers above, since stopping the program with printing a backtrace works fine for me as a defensive programming technique.
If there is a way for assert routine (absolutely no matter how it is syntaxically written and if the word "assert" is used or exists in the programming language or dsl at all) to be called that means some work should be done and the product immediately goes back from the "released, usage-ready" to "needs patch" -- now either rewrite it to real exception handling or fix a bug that caused wrong data to appear.

I mean Assert is not a thing that you should live with and call frequently -- it's a stop signal that indicates that you should do smth to make it never happen again. And saying that "release build should not have asserts" is like saying "release build should not have bugs" -- dude, it's almost impossible, deal with it.
Or think about them as about "failing unittests made and run by end user". You can't predict all the things user is gonna do with your program but if smth too serious goes wrong it should halt -- this is similar to how you build build pipelines -- you halt the process and don't publish, do you? Assertion forces user to stop, report and wait for your help.

Answer 7

It’s pointless to argue what assert should do, it does what it does. If you want something different, write your own function. For example, I have Assert which stops in the debugger or does nothing, I have AssertFatal which will crash the app, I have bool functions Asserted and AssertionFailed which assert and return the result so I can both assert and handle the situation.

For any unexpected problem you need to decide what is the best way for both developer and user to handle it.

Answer 8

As others pointed out, assert is kind of your last bastion of defense against programmer mistakes that should never happen. They're sanity checks that should hopefully not be failing left and right by the time you ship.

It's also designed to be omitted from stable release builds, for whatever reasons the developers might find useful: aesthetics, performance, whatever they want. It's part of what separates a debug build from a release build, and by definition a release build is devoid of such assertions. So there's a subversion of the design if you want to release the analogical "release build with assertions in place" which would be an attempt at a release build with a _DEBUG preprocessor definition and no NDEBUG defined; it's not really a release build anymore.

The design extends even into the standard library. As a very basic example among numerous, a lot of implementations of std::vector::operator[] will assert a sanity check to make sure you're not checking the vector out of bounds. And the standard library will start to perform much, much worse if you enable such checks in a release build. A benchmark of vector using operator[] and a fill ctor with such assertions included against a plain old dynamic array will often show the dynamic array being considerably faster until you disable such checks, so they often do impact performance in far, far from trivial ways. A null pointer check here and an out of bounds check there can actually become a huge expense if such checks are being applied millions of times over every frame in critical loops preceding code as simple as dereferencing a smart pointer or accessing an array.

So you're most likely desiring a different tool for the job and one that isn't designed to be omitted from release builds if you want release builds that perform such sanity checks in key areas. The most useful I personally find is logging. In that case, when a user reports a bug, things become a lot easier if they attach a log and the last line of the log gives me a big clue as to where the bug occurred and what it might be. Then, upon reproducing their steps in a debug build, I might likewise get an assertion failure, and that assertion failure further gives me huge clues to streamline my time. Yet since logging is relatively expensive, I don't use it to apply extremely low-level sanity checks like making sure an array is not accessed out of bounds in a generic data structure. I use it in more high-level contexts with more information specific to the domain of the application.

Yet finally, and somewhat in agreement with you, I could see a reasonable case where you might actually want to hand testers something resembling a debug build during alpha testing, for example, with a small group of alpha testers who, say, signed an NDA. There it might streamline the alpha testing if you hand your testers something other than a full release build with some debugging info attached along with some debug/development features like tests they can run and more verbose output while they run the software. I have at least seen some big game companies doing things like that for alpha. But that's for something like alpha or internal testing where you're genuinely trying to give the testers something other than a release build. If you're actually trying to ship a release build, then by definition, it should not have _DEBUG defined or else that's really confusing the difference between a "debug" and "release" build.

Why has this code to be removed before release? The checks are not much of a performance drain and if they fail there is definitely a problem that I'd prefer a more direct error message about.

As pointed out above, the checks are not necessarily trivial from a performance standpoint. Many are likely trivial but again, even the standard lib uses them and it could impact performance in unacceptable ways to many people in many cases if, say, random-access traversal of std::vector took 4 times as long in what's supposed to be an optimized release build because of its bounds checking that's never supposed to fail.

In a former team we actually had to make our matrix and vector library exclude some asserts in certain critical paths just to make debug builds run faster, because those asserts were slowing down the math operations by over an order of magnitude to the point where it was starting to require us to wait 15 minutes before we could even trace into the code of interest. My colleagues actually wanted to just remove the asserts outright because they found that just doing that made a whopping difference. Instead we settled on just making the critical debug paths avoid them. When we made those critical paths use the vector/matrix data directly without going through the bounds-checking, the time required to perform the full operation (which included more than just vector/matrix math) reduced from minutes down to seconds. So that's an extreme case but definitely the asserts are not always negligible from a performance standpoint, not even close.

But also it's just the way asserts are designed. If they didn't have such a huge performance impact across the board, then I might favor it if they were designed as more than a debug build feature or we might use vector::at which includes the bounds checking even in release builds and throws on out of bounds access, e.g. (yet with a huge performance hit). But currently I find their design a lot more useful, given their huge performance impact in my cases, as a debug-build-only feature which is omitted when NDEBUG is defined. For the cases I've worked with at least, it makes a huge difference for a release build to exclude sanity checks that should never actually be failing in the first place.

vector::at vs. vector::operator[]

I think the distinction of these two methods gets at the heart of this as well as the alternative: exceptions. vector::operator[] implementations typically assert to make sure that out of bounds access will trigger an easily-reproducible error when trying to access a vector out of bounds. But the library implementers do this with the assumption that it won't cost a dime in an optimized release build.

Meanwhile vector::at is provided which always does the out of bounds check and throws even in release builds, but it has a performance penalty to it to the point where I often see far more code using vector::operator[] than vector::at. A lot of the design of C++ echoes the idea of "pay for what you use/need", and a lot of people often favor operator[], which doesn't even bother with the bounds checking in release builds, based on the notion that they don't need the bounds checking in their optimized release builds. Suddenly if assertions were enabled in release builds, the performance of these two would be identical, and usage of vector would always end up being slower than a dynamic array. So a huge part of the design and benefit of assertions is based on the idea that they become free in a release build.

release_assert

This is interesting after discovering these intentions. Naturally everyone's use cases would be different, but I think I would find some use for a release_assert which does the check and will crash the software showing a line number and error message even in release builds.

It would be for some obscure cases in my case where I don't want the software to gracefully recover as it would if an exception is thrown. I would want it to crash even in release in those cases so that the user can be given a line number to report when the software encountered something that should never happen, still in the realm of sanity checks for programmer errors, not external input errors like exceptions, but cheap enough to be done without worrying about its cost in release.

There are actually some cases where I would find a hard crash with a line number and error message preferable to gracefully recovering from a thrown exception which might be cheap enough to keep in a release. And there are some cases where it's impossible to recover from an exception, like an error encountered upon trying to recover from an existing one. There I'd find a perfect fit for a release_assert(!"This should never, ever happen! The software failed to fail!"); and naturally that would be dirt cheap since the check would be performed inside an exceptional path in the first place and wouldn't cost anything in normal execution paths.

Answer 9

Asserts are best used as a tool for testing. I think what the other answers are missing is an example, so here is one:

Let's say you've created some complex data structure that cannot be easily printed of examined with a debugger. You want that data structure to be populated with some data at the start of the program. Then you would write:

for( auto elem : elements ) {
    container.push(elem.index, elem);
    assert(container[elem.index] == elem);
}

That way you can easily test that the code you wrote is doing exactly what you expect it to do. You obviously don't want to check that a == a in a production code but during development it's incredibly useful.

As someone else said, assertions were a prototype of unit testing before unit testing was invented. Therefore assertion should be used the same way unit tests are used. They are there to help you find bugs and ensure that the assertion is always satisfied in your code. If you are shipping code that can have unsatisfied assertions, it means you haven't tested it enough (or used assertion when runtime error checking would be more appriopriate).

If you don't mind the performance penalty of sanity checks (in most cases you are probably right, I'd worry more about useless boilerplate than performance), you can just use if and throw to always check if your pointers are valid. Reserve assert for checks that can be provably true with static analysis.