When are try/exceptions not an anti-pattern?

Question

I just finished a discussion with colleagues regarding the use of exceptions in code. Here is the code (pseudocode) that they had written:

resp = fetch(URL)
if resp.status_code != 200:
     return False

I suggested that they use the below (pseudocode):

try:
    resp.fetch(URL)
except HTTPError as e:
    logging.info("HTTP request failed with error: ", e)
    return False

To my surprise, I was met with a lot of resistance. Evidently, this team has a policy that they almost never use try/except to manage control flow (I am new to the team). I understand that is an anti-pattern in some cases, but here we are actually handling an exceptional (namely, 4xx or 5xx) response. Also, this is a convention that I have seen used in practically every place I've worked so far.

Does my suggest count as an anti-pattern use of try/except? When is it a good, or bad, idea? I realize that there are a number of posts regarding this topic, and I have read through them; I haven't seen anything specific, though, regarding HTTP request/response handling, which I why I post this question.

Answer 1

Alternative perspective - focus on the exceptions and what they represent to the exception's source, rather than try..except.

Eric Lippert in his Blog describes 4 broad categories of Exceptions, summarised below.

Fatal

• Definition: Fatal exceptions are not yor fault, and you cannot sensibly clean up from them.
• Examples: Out of memory, thread aborted.
• Resolution: Don’t catch; let them crash the program.
• Design: Don’t ever throw fatal exceptions directly.

Boneheaded

• Definition: Boneheaded exceptions are violations of the API, and are bugs in your code.
• Examples: Argument is null, index out of range.
• Resolution: Don’t catch; fix them in the code.
• Design: Use code contracts for boneheaded exceptions; do not document the specific exception type.

Vexing

• Definition: Vexing exceptions are due to bad design decisions, thrown in non-exceptional situations.
• Examples: Parsing errors.
• Resolution: Avoid calling vexing functions; if not possible, catch the vexing exception.
• Design: Don’t ever throw vexing exceptions.

Exogenous

• Definition: Exogenous exceptions are from unpredictable, external influences.
• Examples: File not found, resource already in use.
• Resolution: Always catch and handle.
• Design: Throw exogenous exceptions as necessary; document the specific exception type.

Obviously this just focuses on the exceptions themselves there are other important considerations around code which uses exceptions, including:

• If any type of exception cannot be gracefully handled and recovered, ensure the full exception information is logged, including as much information around it as possible.
• If a framework you're using doesn't record unhandled exceptions, include a top-level 'catch-all' near main which can generate a crashdump with that information.
• Use unit tests to assert when exceptions are expected to be thrown and when they are expected to be handled.

Answer 2

Try/except is not an antipattern, although it is probably an antipattern to turn an error condition into a boolean. But this happens in both examples, so that is a separate question.

Both status codes and exceptions are valid ways to handle HTTP error responses. The choice is a question of abstraction level.

If you are writing networking-level code (e.g. a proxy server), you probably want to treat all status codes on the same level of abstraction. A 4xx or 5xx status does not mean your program cannot fulfill its task, it is just a response which have to be handled appropriately.

If you are writing application level code, e.g some business logic which perform HTTP requests to fetch data, you want to separate network-level error conditions from the regular application logic, and exceptions are appropriate.

There are two separate questions: When is it appropriate to throw an exception, and when it is appropriate to catch an exception?

It is appropriate to throw an exception when a condition occurs which cannot be meaningfully handled at the level of abstraction of the current function. Of course this requires functions to be clear about abstraction levels, which might be the harder problem. Consider a function called sendHttpRequest(url, method, payload). A HTTP response code would be at the same level, so it would be reasonable to have as part of the return value.

But consider the function getCurrentExchangeRate(fromCurreny, toCurrency) which internally calls some HTTP service. Clients calling this method would not be expected to know about HTTP response codes. It would therefore be appropriate for the function to throw an exception if it for whatever reason is not able to return the exchange rate.

Exceptions should only be caught when code is able to meaningfully handle the condition. If an exception cannot be handled at any level, then it should be allowed to rise to the top and terminate the application (fail fast principle).

Handling an exception means the code should be able to continue without fear of any state being corrupted by the operations which were not completed. (For example, returning 0 for the missing exchange rate would not be an appropriate way to handle the error, since this just leads to data corruption down the line. If in doubt, always fail fast.)

Catching an exception and returning false would almost never be a useful way to handle an exception or error condition, since it doesn't provide any information about what went wrong, and therefore no reasonable way for callers to handle the problem. A HTTP request could fail because of intermittent network errors, or it could fail because a hardcoded URL has a typo. These errors should be handled differently - network errors could be handled by re-trying the request, while the typo should fail fast, alerting developers to the problem.

Answer 3

To understand why it's bad you have to take a holistic view of the calling code and the function together.

func doThing()
{
    try:
        resp.fetch(URL)
        return true;
    except HTTPError as e:
        logging.info("HTTP request failed with error: ", e)
        return False
}

main() 
{
    var result = doThing();
    if(result)
    {
        //continue with other things
    }
    else
    {
        //log an error and move to some other branch?
    }
}

Now consider a simplified version:

func doThing()
{
    resp.fetch(URL)
}

main() 
{
    try:
        doThing(); 
        //continue with other things           
    except Error as e:
        logging.error("Error happened!", e)        
}

You can see the second version is just better. You have less conditions and if exceptions are exceptional you could even get rid of the try catch and have some sort of global error handler.

I would sum up the general rules as follows

1. Don't catch and return false when you could let the exception bubble up instead.
2. Don't log in low level functions, let the exception bubble and log at the top level.
3. Don't swallow exceptions and continue unless the failure is expected. ie. a known bad network connection, or you have a list of things and you know some will fail.

Second question. Is the original function your colleagues wrote good?

resp = fetch(URL)
if resp.status_code != 200:
    return False

Here, unlike in your code, it looks like fetch won't throw an exception when non success codes are returned. so they have added this check and return false.

However, if we look at the calling code again, that means we have to check the result is true everytime, leading to lots of needless conditional statements.

It would be better to throw an exception instead which allows the more simple way of calling the function.

MyObject DoThing()
{
    resp = fetch(URL)
    if resp.status_code != 200:
        throw "Error connecting to resource"
    return ResultParser.Parse<MyObject>(resp.body);  //whatever code you need to parse the response
}

Answer 4

First of all, most every modern http client library has a way to raise exceptions based on status code. Even hardcore functional programming libraries like http4s provide an API that raises an error in your IO monad instead of forcing an explicit check. If it were an anti-pattern, it wouldn't be so prevalent.

Exceptions exist in the first place because most production code has one "happy path" and one or more other paths. These are separate concerns that are easier to code if you have a mechanism like exceptions that allows you to separate them, especially if you are composing several steps that each have multiple paths.

For exceptions to be beneficial, you need to actually use them to separate concerns. If you're just setting a boolean whether your request succeeds or fails, the exception doesn't help you. You'd have to restructure some of the calling code so handling non-2xx status codes is consolidated in one place, and handling json parse errors of the response body is consolidated in one place, and application-level errors are consolidated in one place, and the happy path code isn't frequently interrupted by error handling code.

That means in general if your try only covers one line of code and your catch is in the same method, the exception isn't usually buying you anything, and it's costing you some code simplicity and some stack unwinding time.

In summary, don't try to think of exceptions as being "rare" things, but as separate concerns that you don't want interrupting your main flow.

Answer 5

In this case I am with your colleague. Not using exception here is cheap, while using it is not. So why would you like to use it?

Currently my design principle is as follows -- is an error a completely surprise to me/code/design? If yes -- use exception, if not -- use "error token".

And currently my error token consists of -- severity, message, source and optionally failure data.

Such design allows to be more forgiving when running code, but strict when running tests, also it is much better to bubble up the most crucial error up.

Answer 6

Use exceptions for exceptional situations. When you request a URL, the following are not exceptional:

1. No internet connection at all.
2. Cannot find the server in question.
3. Cannot establish a secure connection to the server.
4. Malformed resource name.
5. Request times out.
6. Status code telling you that the server made a request timing out.
7. Resource not found.
8. No permission to access resource
9. Missing authentication for resource
10. User cancelled the operation
11. Operation wasn’t allowed to proceed because it required a user interaction.

and many others.

MacOS and iOS distinguish between errors (anything happened that stopped you from getting a response from the server), and status codes (client received a status code from the server).

It is usually best to have one method that attempts a download, handles any problems in a suitable way, and returns either success with the data downloaded, success with no data downloaded, or failure, so all complex error handling needs to be written only once.

Answer 7

It really depends on the context/design. A non-200 status doesn't necessarily have to raise an exception in your code unless it's designed that way. Regardless, it's still possible to get an exception for something else like no network connection, or the request timed out. You can either handle those in your code, or they may be getting handled elsewhere and by design your code is expected to ignore handling them.

You don't mention it, but this looks like Python. Python 3.11 came out with "zero cost" exception handling1, so if part of your colleagues reason is performance, maybe that can be less of an issue in the future?

I'd probably try to do something like this:

try:
    resp.fetch(URL)
    if resp.status_code == "":
        # Make sure that there is/was a status code
        raise NoStatusCodeException
    elif resp.status_code != 200:
        return False
except KnownException as e:
    # Known issue that I want to generally ignore and still handle with return False
    logging.debug("Return False, ignore KnownException: ", e)
    return False
except Exception as e:
    # Unexpected issue that should be reviewed
    logging.error("Unexpected exception: ", e)
    return False

Answer 8

HTTP was not made with exceptions in mind. It predates the OO-era. It is pointless to map HTTP status codes to exceptions. None of them can be considered "exceptional" or errors in the sense that something went wrong in your program. So I don't think exception handling is helpful here, it is just confusing and mixes mutually exclusive paradigms.