Handling exceptions I don't know about

Question

When I work with handling exceptions, I notice that I often have to deal with the ones I had no idea about. Especially it is noticeable when I program a method that grabs data from web. An error may occur, for example, due to connection loss, I can handle it. But then another error occurs, a different error with the same cause - connection loss. Okay, added it. Sometimes even yet another occurs. So, the problem is, I am never sure enough if I've handled all the possible errors that may occur due to a certain cause.

At first, I thought about going wildcard and do something like (example in Python):

try:
    #do stuff
except:
    #handle error

But it proved to be a wrong approach soon, cause if I need to handle, say, KeyboardInterrupt, which is raised when a user terminates the program, instead of being handled by a scope I want it to be handled by, it is handled by this wildcard, which is not supposed to have anything to do with it.

So how do I handle exceptions I don't know of but that may possibly occur (or not occur)? Some kind of exceptEverythingBut KeyboardInterrupt:? I doubt many languages have that in their syntax.

EDIT1: a really simplified example:

#!/usr/bin/python3 -u
# -*- coding: utf-8 -*-

try:
    while True:
        try:
            print(1)
        except:
            print(2)
except KeyboardInterrupt:
    print('end')

When I press Ctrl+C, I want it to print 'end' and finish. But instead it prints 2 and continues execution.

If I try this:

#!/usr/bin/python3 -u
# -*- coding: utf-8 -*-

try:
    while True:
        try:
            print(1)
        except KeyboardInterrupt:
            break
        except:
            print(2)
except KeyboardInterrupt:
    print('end')

it finishes, but it skips the outer except and doesn't print 'end'. And that's not what I want. So, the only way I see is to prevent the inner scope from handling KeyboardInterrupt altogether. But it is not possible if there is a except: or except KeyboardInterrupt: in there. So, I need to specify exactly which errors I want to handle in the inner except. But, as I mentioned in the beginning, I don't always know what they can be.

I'm asking this question, because my common way to do it is to just let the program unexpectedly fail several times, read the logs and add handling of errors I didn't know about to new versions; however this could just be a naïve approach, so I want to know how it is done by experienced people.

Answer 1

Just rethrow the exception when you get it:

try:
  #do stuff
except KeyboardInterrupt:
  raise
except:
  #do other stuff

You specify that you get an exception that you know that you want to handle differently than default case, but you don't know how to handle it - so you just pass it, by throwing it again.

Some working example:

#!/usr/bin/python2

if __name__ == "__main__":
    try:
        while True:
            try:
                print(1)
            except KeyboardInterrupt:
                raise
            except:
                print(2)
    except KeyboardInterrupt:
        print('end')

Please note that you shouldn't use except:, as this is a bad practice. except: used in this example is used in sole purpose to make things as simple as possible.

Answer 2

In general, you should not use except:, except for a few extremely rare circumstances, most notably log-and-reraise, or at the very top-level after which the program must end. Do not use it in libraries, or any non-top-level method, at all.

Instead, please use except Exception:. It catches most 'normal' errors, except a few special ones (KeyboardInterupt, 'SystemExit`, and some others). See https://docs.python.org/3.6/library/exceptions.html#exception-hierarchy.

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- ... (all other exceptions)

```

In most cases, I would even recommend against using except Exception:, and instead chose the most specific exception type(s) possible. This is because things like NameError, AttributeError, KeyError and the like are all subclasses of Exception. You can hide bugs (typos) in a very subtle way with this.

When dealing with network connections, you're most likely to want to use something like except socket.error:, or except LibrarySpecificErrorHere. Check the exception hierarchy for the networking library you use. You don't necessarily have to use the leaf exception classes.

With a wider except clause, more bugs will be hidden by it. Therefore: have as little code as possible in the try block, and have the exception classes as specific as possible.

Consider this:

bars = []
for foo in foos:
    try:
        bar = foo.bar
    except AttributeError:
        # Sometimes we get a `foo` that does not have a `bar` set yet.
        pass
    else:
        bar.apend(bar)

versus

bars = []
for foo in foos:
    try:
        bar = foo.bar
        bars.apend(bar)
    except AttributeError:
        # Sometimes we get a `foo` that does not have a `bar` set yet.
        pass

Did you notice the spelling error in append (I 'accidentally' wrote apend). You are smart, so you did notice. But Python misses it in the second case, while in the first case the scope of the exception handling is smaller, and thus a bug was lurking outside the scope of the handling, got out, and you saw it.

The same goes for this piece of code:

bars = []
for foo in foos:
    try:
        bar = fo.bar
    except:
        # Sometimes we get a `foo` that does not have a `bar` set yet.
        pass
    else:
        bars.append(bar)

Oopsie, did I just accidentally ignore a NameError?

Again: except: catches all exceptions, even the ones you are not supposed to catch (SystemExit, KeyboardInterupt, ...). In general, the application is supposed to be dead after any of those exceptions are raised. Be very specific in handling exceptions, make sure as few bugs as possible get hidden by overly aggressive exception handling.

Answer 3

If you assign a custom error handling function to sys.excepthook, it will be called whenever an exception is raised and you won't need to use a try-except statement. Something like this should work:

import sys

def on_error(etype, value, tb):
    if etype == KeyboardInterrupt:
        return
    # handle other types of errors

sys.excepthook = on_error

Answer 4

And this is why I like Java - no such surprises there (every function explicitly tells you what Exceptions it might throw).

Wouldn't that be something like this?

try:
    #do stuff
except(KeyboardInterrupt):
    pass
else:
    #handle error