Why do many exception messages not contain useful details?

Question

It seems there is a certain amount of agreement that exception messages should contain useful details.

Why is it that many common exceptions from system components do not contain useful details?

A few examples:

• .NET List index access ArgumentOutOfRangeException does not tell me the index value that was tried and was invalid, nor does it tell me the allowed range.
• Basically all exception messages from the MSVC C++ standard library are utterly useless (in the same vein as above).
• Oracle exceptions in .NET, telling you (paraphrased) "TABLE OR VIEW not found", but not which one.

So, to me it seems that for the most part exception messages do not contain sufficient details to be useful. Are my expectations out of line? Am I using exceptions wrong that I even notice this? Or maybe my impression is wrong: a majority of exceptions do actually provide useful details?

Answer 1

Exceptions do not contain useful details because the concept of exceptions has not matured yet enough within the software engineering discipline, so many programmers do not understand them fully, and therefore they do not treat them properly.

Yes, IndexOutOfRangeException should contain the precise index that was out of range, as well as the range that was valid at the time that it was thrown, and it is contemptible on behalf of the creators of the .NET runtime that it doesn't. Yes, Oracle's table or view not found exception should contain the name of the table or view that was not found, and again, the fact that it does not is contemptible on behalf of whoever is responsible for this.

To a large part, the confusion stems from the misguided original idea that exceptions should contain human-readable messages, which in turn stems from a lack of understanding of what exceptions are all about, so it is a vicious cycle.

Since people think that the exception should contain a human-readable message, they believe that whatever information is carried by the exception should also be formatted into the human-readable message, and then they are either bored to write all the human-readable message-building code, or they are afraid that doing so might be divulging an inadvisable amount of information to whatever prying eyes might see the message. (The security issues mentioned by other answers.)

But the truth of the matter is that they should not be worrying about that because the exception should not contain a human-readable message. Exceptions are things that only programmers should ever see and/or deal with. If there is ever a need to present failure information to a user, that has to be done at a very high level, in a sophisticated manner, and in the user's language, which, statistically speaking, is unlikely to be English. (¹)

So, for us programmers, the "message" of the exception is the class name of the exception, and whatever other information is pertinent to the exception should be copied into (final/readonly) member variables of the exception object. Preferably, every single conceivable little bit of it. This way, no message needs to (or should) be generated, and therefore no prying eyes can see it.

To address the concern expressed by Thomas Owens in a comment below:

Yes, of course, at some level, you will create a log message regarding the exception. But you already see the problem with what you are saying: on one hand, an exception log message without a stack trace is useless, but on the other hand, you don't want to let the user see the entire exception stack trace. Again, our problem here is that our perspective is skewed by traditional practices. Log files have traditionally been in plain text, which may have been fine while our discipline was in its infancy, but perhaps not any more: if there is a security concern, then the log file must be binary and/or encrypted.

Whether binary or plain text, the log file should be thought of as a stream into which the application serializes debug information. Such a stream would be for the programmers' eyes only, and the task of generating debugging information for an exception should be as simple as serializing the exception into the debug log stream. This way, by looking at the log you get to see the exception class name, (which, as I have already stated, is for all practical purposes "the message",) each of the exception member variables which describe everything which is pertinent-and-practical-to-include-in-a-log, and the entire stack trace. Note how the formatting of a human-readable exception message is conspicuously missing from this process.

P.S.

A few more of my thoughts on this subject can be found in this answer: How to write a good exception message

P.P.S.

It appears that a lot of people were being ticked off by my suggestion about binary log files, so I amended the answer once again to make it even more clear that what I am suggesting here is not that the log file should be binary, but that the log file may be binary, if need be.

---

¹ Any notion of internationalizing exception messages is deeply misguided. This is how we arrive at preposterous situations such as this one, which just happened to me, where we collected a log from the field, (China to be specific,) to find this nonsense in it:

Caused by System.DllNotFoundException : 无法加载 DLL“libMathNetNumericsMKL”: 找不到指定的模块。 (异常来自 HRESULT:0x8007007E)。

(According to Google Translate, 无法加载 means "Unable to load", 找不到指定的模块 means "The specified module could not be found", and 异常来自 means "The exception comes from".)

Answer 2

Why is it that many common exceptions from system components do not contain useful details?

In my experience, there are a number of reasons that exceptions do not contain useful information. I expect that these sorts of reasons would also apply to system components - but I don't know for sure.

• Security focused people see exceptions as a source of information leakage (for example). Since the default behavior of exceptions is to display that information to the user, programmers sometimes err on the side of caution.
• In C++, I've heard arguments against allocating memory in catch blocks (where at least some of the context to make good messages lies). That allocation is hard to manage, and worse, can cause an out of memory exception there - often crashing your app or leaking memory. It's hard to format exceptions nicely without allocating memory, and that practice may have migrated between languages as programmers do.
• They don't know. I mean, there are scenarios where the code has no idea what went wrong. If the code doesn't know - it can't tell you.
• I've worked at places where localization concerns prevented putting English only strings into the system - even for exceptions that would only be read by the English speaking support staff.
• Some places, I've seen exceptions used more like asserts. They're there to provide a clear, loud message during development that something isn't done, or a change has been done in one place, but not another. These are often unique enough that a good message would be duplicated effort or simply confusing.
• People are lazy, programmers more than most. We spend far less time on the exceptional path than the happy path, and this is a side effect.

Are my expectations out of line? Am I using Exceptions wrong that I even notice this?

Kinda? I mean, exceptions should have nice messages, but they're also exceptions. You should be spending your time designing code to avoid exceptional conditions, or writing code to handle exceptional conditions (ignoring the message), not using them as a sort of interactive feedback mechanism when coding. It's unavoidable to use them for debugging purposes, but in most situations that should be kept to a minimum. That you notice this problem makes me concerned that you're not doing a good enough job at preventing them.

Answer 3

I don't have an excess of C# experience, or C++ specifically, but I can tell you this - developer-written exceptions 9 out of 10 times are more useful than any generic exception you will ever find, period.

Ideally yes, a generic exception will point you to exactly why the error occurred and you'll be able to fix it with ease - but realistically, in large applications with multiple classes that can throw a wide variety of different kinds of exceptions, or the same kind of exceptions, it is always more valuable to write your own output for error return than it is to rely on the default message.

This is how it should be, because as many people have pointed out, some applications don't want to throw an error message that they don't want the user to see, for security reasons or to avoid confusing them.

Instead, you should anticipate in your design what types of errors might be thrown in your application (and there will always be errors) and write error-catching messages that help you identify the issue.

This won't always help you - because you can't always anticipate what error message will be useful - but it is the first step to understanding your own application better in the long-run.

Answer 4

The question is specifically asking why do so many exceptions thrown by "system components" (aka standard library classes) not contain useful details.

Unfortunately, most developers do not write the core components in standard libraries, nor are detailed design documents or other design rationale necessarily made public. In other words, we may never know for sure.

However, there are two key points to keep in mind as to why detailed exception information might not be desirable or important:

1. An exception may be used by calling code in any way: a standard library cannot place constraints on how the exception is used. Specifically, it may be displayed to the user. Consider an array index out of bounds: this may give useful information to an attacker. The language designer has no idea how an application will use the thrown exception or even what type of application it is (e.g. web app or desktop), so leaving out information may be safer from a security perspective.

2. Exceptions should not be displayed to the user. Instead, display a friendly error message and log the exception to a location an attacker has no access (if applicable). Once the error is identified, a developer should debug through the code, inspecting stack frames and logic paths. At this point, the developer has more information than an exception could ever hope to have.

Answer 5

First off, let me burst a bubble by saying even if the diag message is loaded with information that brings you to the exact code line and sub command in 4 seconds, chances are the users will never write it down or convey it to the support folks and you will be told "Well it said something about a violation... I don't know it looked complicated!"

I've been writing software and supporting the results of others software for more than 30 years now, and personally, the current quality of an exception message has virtually nothing to do with security, regardless of how the end results happened to fit their model of the universe, an much more to do with the fact that so many in our industry were originally self taught, and they just never included lessons in communications. Maybe if we FORCED all new coders into a maintenance position for a couple of years where they had to deal with figuring out what went wrong, they would understand the importance of at least some form of precision.

Recently in an application being rebuilt, a decision was made that return codes would fall into one of three groups:

• 0 through 9 would be success through success with additional information.
• 10 through 99 would be non fatal (recoverable) errors, and
• 101 through 255 would be fatal errors.

(100 was for some reason left out)

In any particular workflow, our thought was reused or generic code would use generic returns (>199) for fatal errors, leaving us with 100 fatal errors possible for a work flow. With slight differentiating data in the message, errors such as file not found could all use the same code and differentiate with the messages.

When the code came back from the contractors, you would not believe our surprise when virtually EVERY SINGLE FATAL ERROR was return code 101.

All that considered I think the answer to your question is the messages are so meaningless because when originally created, they were to be placeholders that no one got back to. Eventually folks figured out how to fix issues not because of the messages but DISPITE them.

Since that time, the self taught folks simply never had a good example of what an exception should contain. Add to that more users don't read the messages, much less try to pass it on to support (I've seen error messages that were cut and paste by the used that were then redacted before being sent on with the later comment that it seemed like a lot of information and I could not possibly want it all, so they randomly removed a bunch of it.

And lets face it, with way too many (not all but way too many) of the next generation of coders, if it is more work and does not add flash, just not worth it...

Last note: If an error message includes an error/return code, it seems to me that somewhere in the executed modules there should be a line that reads something like "if condition return code-value" and condition should tell you why the return code occurred. This seems like a simple logical approach, but for the life of me, just TRY to get Microsoft to tell you what happened when a windows upgrade failed on CODE 80241013 or some other very unique identifier. Kinda sad, isn't it?

Answer 6

While I do agree that exceptions should contain as much information as possible, or at least be less generic. In the table not found case, the table name would be nice.

But you know more about what you were trying to do at the place in the code where you received the exception. While you often really can't do much to rectify the situation when something goes wrong in a library outside your control, you can add much more useful information on in which situation something has gone wrong.

In the case of table not found, it will not help you much if you are told that the table that cannot be found is called STUDENTS, because you do not have such a table, and that string is nowhere in your code.

But if you catch the exception and rethrow it with the SQL-statement you were trying to execute, you will be better off, as it turns out that you tried to insert a record with the name field being Robert'); DROP TABLE STUDENTS; (There is always an xkcd!)

So to combat the less than informative exceptions: try-catch-rethrow with more information specific to what you were trying to do.

I should probably add, for this to be more of an answer as to the why in the question, that a likely reason why the focus from the makers of the libraries has not been on making the exception messages better, is that they do not know why something has been tried that failed, that logic is in the calling code.

Answer 7

To give a slightly different answer: The offending code has probably been done to spec:

• The function receives X and returns Y
• If X is invalid, throw exception Z

Add the pressure to deliver exactly to spec (for fear of being rejected in review/testing) in minimum time and with minimum fuss, then you have your recipe for an entirely compliant and unhelpful library exception.

Answer 8

Exceptions have a language- and implementation- specific cost.

For example, C++ exceptions are required to destroy all the living data between throwing call frame and catching call frame, and that is expensive. Hence, programmers do no wish to use exceptions a lot.

In Ocaml, exception throwing is nearly as fast as a C setjmp (its cost does not depend upon the number of traversed call frames), so developers can use it a big lot (even for non-exceptional, very common, cases). In contrast, C++ exceptions are heavy enough so you probably won't use them a lot like you would in Ocaml.

A typical example is some recursive search or exploration which can be "stopped" inside a quite deep recursion (e.g. find a leaf in a tree, or a unification function). In some languages it is quicker (so more idiomatic) to propagate that condition to every caller. In other languages, throwing an exception is quicker.

So depending on the language (and the habits of developers using it), an exception could carry a lot of useful details, or on the contrary be used as a quick non-local jump and carry only the very useful data.

Answer 9

TL;DR: No code should ever interrogate an Exception's Message property.

Why do many exception messages not contain useful details?

Arguably, because none of them should!

Exception Objects should carry as much information as is useful for an Exception Handler to do its job, i.e. to "handle" the Exception and make it "go away".

The class of an Exception and, in some technologies, its properties can be used to selectively catch an Exception for handling. Why is there no such mechanism that can be used with the Exception's Message? Because the designers of Structured Exception Handling architectures realised this, simple truth ...

Exception Messages are largely redundant Documentation, intended for people, not programs.

Exception Messages are there largely for the Application's "back-stop" Exception Handler, way up at the very top of the Call Stack, that catches the Exception and logs it away for diagnostic purposes just before the Application crashes and burns.

Having a big, headline Error Message in the Log File makes it easy to spot.

Exception Messages also have a nasty habit of being edited, re-worded, shuffled-about-a-bit or translated by Developers who think that the existing wording is clunky or meaningless or overly-wordy or technically inaccurate or 101 other reasons and, when they do so, any code that expects to find particular "bits" of "useful" stuff embedded in the Message suddenly stop working.

Answer 10

What makes you think the index value or the required range or the name of the table is a useful detail, for an exception?

Exceptions aren't an error handling mechanism; they are a recovery mechanism.

The point of exceptions is to bubble up to the level of code that can handle the exception. Wherever that level is, either you have the information needed, or it is not relevant. If there is information you need, but don't have immediate access to, you are not handling the exception at the appropriate level.

The one time I can think of where extra information could be helpful is at the absolute top level of your application where you crash and emit an error dump; but it is not the job of the exception to access, compile and store this information.

I'm not saying that you can just put "throw new Exception" everywhere and call it good, it is possible to write bad exceptions. But including irrelevant information is not nessecary to use them properly.

Answer 11

Most programs have two requirements:

1. Behave usefully when practical.

2. When unable to behave usefully, behave in tolerably useless fashion.

In many cases, exceptions are used as a means of meeting the second requirement as cheaply as possible. Having a program report why it is unable to perform its primary task may be somewhat more useful than having it simply refuse to perform the task without stating a reason, but such benefit might not be worth the cost of trying to provide more detailed information.

Note that this philosophy is rather different from that of "modern" compilers based on gcc or LLVM, which take the attitude that if a program can't perform usefully, all possible behaviors should be regarded as equally satisfactory. Exception handling is designed around the presumption that the marginal value in ensuring that attempts to process erroneous data will be acceptably harmless is larger than the marginal value of making it easy to respond to such data somewhat usefully.

Answer 12

I have found it very useful to divide the concept into two extremes and a grey area in-between.

• Error are intended to state that execution cannot continue because something went wrong. Errors are non-recoverable, and are intended for consumption by an end user to diagnose the issue (perhaps restricting that to a "developer" end user in security relevant settings)
• Exceptions are intended to provide exceptional code flow. They are designed to provide a path of execution through the code which permits recovery and continuation of execution.
• Unhanded Exceptions are the grey area. They are exceptions, in that they were intended to provide exceptional code flow but were not handled as such. Thus they act like Errors.

A key question with exception handling is "can you recover." Is this particular exception recoverable? In a multi-request setting like a web server, where there are many parallel unrelated requests being processed, often one can recover simply by catching the exception, providing an error message to the user for this request, and continuing to process other requests. However, what if the exceptional code flow left a database locked so that other processes cannot access it? What if you have a traditional banker-app example and you've deposited $100 in one account without debiting the other $100? These are cases where one should not undertake recovery.

So what information needs to be in an exception? If the exception is raised/thrown in the function that is called, then it should have enough information for the caller to figure out what went wrong. But what if it gets called in the a function the callee calls? What if my "getDocument" function calls the language-specific "openFile" function which throws an error. If the error is FileNotFound, then it may be obvious that the information needed by getDocument to solve the problem (if it wanted to) is the same information needed by its caller. But if the error is EncodingFailure because one needed to convert a unicode string into an OS specific string format, it is very unlikely that getDocument would need the same information as its caller -- its caller would need to know something about why getDocument was encoding in the first place.

Many languages now offer "inner exceptions" for this purpose. They catch the exception and raise/throw a new exception that is linked to the original exception. Perhaps "getDocument" has a FileOpenError exception that it can throw, and the inner exception might be "EncodingFailure". "getDocument" can make the statement that the result of any call which raises FileOpenError is nullimpotent, which is the extra information you need to know to be sure that it's safe to continue executing (the application will not be left in an illegal state). This "inner exception" pattern is the closest I have seen to implementing the ideal you mention of providing "all" of the data -- particularly all of the data that should be relevant to recovery.

We, as an industry as a whole, are still working on what exceptions really mean. Each language I have seen implement it evolves the idea ever so slightly.