Testing generic errors

Question

I've got a program that validates it's input, and then errors.

So far for each kind of error I created a new derived class. But this was getting pretty unwieldy (the input is complex so there are many possible errors) so instead I made the base more flexible and removed all the derived classes.

The core resulting issue is that now when I test with certain invalid inputs, I'm not sure how to identify the "correct" error anymore. If the program produces an error on invalid input but it's the wrong one, how am I going to identify the correct error without effectively hardcoding internal error implementation details?

Edit: I also intended to specify that the reason I went with a new derived class per error type is that I need dynamically extensible error types. Something like an enumeration would never be suitable.

Answer 1

I believe the standard approach is to use an error code (of enum type) and an error message (of string type). The "code" makes it easy to programmatically identify a specific type of error (if(e.code === DIVIDE_BY_ZERO)), and the "message" makes it easy to show something meaningful to the user (std::cout << e.message).

A new derived exception class should probably be reserved for those categories of exceptions that you know you want to handle differently from the others. In other words, if you know of a place where you have code you want to put in a catch(const SomeNewExceptionType& e) block, then it's worth creating SomeNewExceptionType. This usually happens when the error handler needs significantly more information than just the type of error. For instance, a syntax exception may want to carry around a chunk of the abstract syntax tree it was working on when something went wrong.

It's also possible that you're simply categorizing your errors wrong (eg, creating MissingSemicolonException and TooManyArgumentsException and so on instead of a single SyntaxException). We may need more information to help if that's the case.

Answer 2

Aside the type, you can usually specify a textual description. For instance, in a method SetPercentage(value) a error OutOfRangeError thrown when the value is inferior to zero, but also superior to one hundred.

Same type—two different errors.

The text of the error can then specify what went wrong specifically. For instance:

if (value < 0) {
    raise OutOfRangeError("The value should not be inferior to 0.");
}

if (value > 100) {
    raise OutOfRangeError("The value should not be superior to 100.");
}

Description used purely for debugging

This is what is commonly used in languages such as Java or C# which use exceptions rather then errors. In those exceptions, the description is intended to be read by a developer maintaining (and debugging) the application; it is not expected to be shown to the end user or to be used programmatically. It is not that useful to be able to determine programmatically which one of the two exceptions occurred (doing a switch over the textual description is obviously a terrible idea).

Look at the larger picture. Imagine that the code above is in our business layer. Then, in presentation layer, we have:

try:
    int percentage = conversions.parseInt(field.value);
    if (percentage < 0) {
        field.highlightInRed();
        // The percentage should be superior to zero.
        form.showError("Le pourcentage doit être supérieur à zéro.");
    }
    else if (percentage > 100) {
        field.highlightInRed();
        // The percentage should be inferior to one hundred.
        form.showError("Le pourcentage doit être inférieur à cent.");
    }

    // Call to the business layer happens here.
    rebate.changeValue(precentage);

catch ConversionError:
    field.highlightInRed();
    // The percentage should be an integer number.
    form.showError("Le pourcentage doit être un nombre entier.");

Here, we don't wait until the business layer throws an exception: an exception should remain exceptional, while users typing "13" when requested to enter a digit or users writing "250" when requested to enter a number between 0 and 100 are not that exceptional.

Interaction designers may even decide that input validation may happen on every key stroke. You type 3 and 8, the field background remains white. You type an additional 0—the field background turns red and a hint appears, showing that 380 is not a valid value for this field. You wouldn't ask the business layer to generate exceptions at every key stroke, would you?

It's only if I fail, as a developer, to provide correct checks at presentation layer, than OutOfRangeError in business layer will be raised. This would probably result in a crash and a bug report containing the stack trace: a person debugging that will then find and solve the error in the presentation layer.

---

If you notice some sort of logic duplication between the business layer and the presentation layer in the example above, refactoring may help:

1. In business layer, value was an integer. What if I create a type Percentage inherited from a newly created class IntInRange:

   abstract class IntInRange
   {
       private int value;
       public abstract readonly IntInRange Min;
       public abstract readonly IntInRange Max;
       public void SetValue(int value)
       {
           // Raise OutOfRangeErrors here.
           this.value = value;
       }
       ...
   }
   
   class Percentage : IntInRange
   {
       public readonly Percentage Min = 0;
       public readonly Percentage Max = 100;
   }
   

2. In presentation layer, I can then use data binding to tell that a given field is associated with the variable rebate.percentage of type Percentage. If the data binding framework is smart, I can bridge its error handling with the rules described in IntInRange type, and the framework will handle all the errors itself. If it's really smart, it will generate humanly-readable text of the errors, such as "The percentage should be superior to 0.", the word "percentage" corresponding to the name of the variable.

Notice that here, the text of the exception is still neither used programmatically, nor shown to the end user. The data binding framework figures the information from the class itself, and the OutOfRangeError should never be hit (or it would indicate a bug in the framework or in the code using this framework).

Code used for APIs

Another usage which is proper to errors, and not exceptions, consists of embedding a code in the error. This code is then used not only during the debugging, but also by third-party developers accessing an API.

For instance, an abstract API can return Error 500371, which appears to indicate that the percentage value is inferior to zero, or Error 500373, which means that the percentage value is superior to one hundred.

While many APIs use cryptic error codes (often numbers), this is not always the case. Especially REST services use more and more text identifiers of an error:

{
    "error": {
        "id": "percentage-superior-to-100",
        "uri": "http://example.com/errors/percentage-superior-to-100",
        "description": "The specified value of the percentage is out of range. The value should be superior or equal to 0 and inferior or equal to 100."
    }
}

Here, while description can change (and can be localized), the id is guaranteed to remain the same forever (until the API is obsoleted) and can be used programmatically.

Answer 3

In C#, the way this is done is by adding metadata to your object's members in the form of property attributes, each of which is a class. There aren't that many of them; they embody categories of errors, and some of them take parameters. For example, to validate a class property as a number between 0 and 50, you might decorate the property with an attribute that looks something like this:

[Range(0, 50)]

And then use Reflection to inspect that metadata to determine how to validate each property.

It is my understanding that C++ isn't as adept at this type of introspection. In the old days of ASP.NET MVC, we didn't have these attributes, so we would simply write a validation method that would return a collection of ValidationError objects. This collection could then be handed back to the View for display to the user. In a ViewModel object, this makes perfect sense, because the only purposes of the ViewModel object is to display data to, and retrieve data from, the user via the UI.

Answer 4

I could also use a template to generate derived classes via tagging rather than actually generating a new class each time. Something like

class BaseException {
    ...
};
template<typename T> struct DerivedException : BaseException {
    using BaseException::BaseException;
};
struct CyclicReference {};

Then I could employ

throw DerivedException<CyclicReference>(...);

This would mean that I could still name/specify each kind of exception, but it would not be infinitely laborious to specify each derived class individually, as it's just a separate "tag" type for each one, which I feel is much more maintainable, and users can still specify their own "tag" types or even just use the base directly. Then during testing, I can simply check the tag. And as a double bonus, I could specialize or create a new derived if I really needed more than the base.

Triple bonus: I've just discovered that re-using the tags almost certainly means duplicated code.