Exception handling in Python - Am I doing this wrong (and why?)

Question

I've read many questions and articles on exception handling in Python (and in general), but I still think that it's the most confusing thing ever. I ended up doing something like this:

# error class for exceptions specific to this script
class MyError(Exception): pass

# error class for exceptions in MyClass,
# which can be used from outside of this script
class MyClassError(Exception): pass

class MyClass:
    ...
    # class has method that takes a function as a parameter
    def method(self, param_function):
        ...
        param_function(a, b, c, d)
        try:
            some_other_funtion(arg)
        except ValueError, e:
            raise MyClassError("Some message")
        ...

# outside of MyClass
if __name__ == "__main__":

    # read input from console etc.
    # ...   

    def function(a, b, c, d):
       # ....
        try:
            # something with a
        except ValueError, e:
            raise MyError("My message:" + a)
        try:
            # something with b
        except ValueError, e:
            raise MyError("My message:" + b)
       try:
            # something with c
        except ValueError, e:
            raise MyError("My message:" + c)
        try:
            # something with d 
        except ValueError, e:
            raise My Error("My slightly different message:" + d)  

    x = MyClass()
    try:
        x.method(function)     
    except (IOError, MyError, MyClassError) as e:
        print e

Explanation:

I want the user of the script to just get a user-friendly message if the error is a result of him passing the wrong input.
The messages with which I raise my custom exceptions are user-friendly, and they (in my opinion) contain all the information I want the user to see.

That's also the reason why I'm handling IOError in a similar manner. I expect an IOError only if the user provides a wrong filename, and I want him to see something like "File not found: name of the file" and nothing more.

Why this seems wrong:

In my main function, in function, I'm basically surrounding every line of code with a try-catch, and handling every exception in a similar manner - sometimes the message the user should see is a little different, but mostly the only difference is the parameter in the message.

Questions:

Am I handling my exceptions in the right places?
Am I handling them in the right way?
Is the code duplication that I'm doing in function neccessary?

Answer 1

User input sucks. You can't trust those users to get anything right, and so you've got to handle all kinds of special cases that make your life difficult. Having said that, we can minimize the difficulty with general principles.

Validate early, not often

Check input for validity as soon as it read into your program. If you read in a string that should be a number, convert it into a number right away and complain to the user if it isn't a number. Any rogue data you don't verify at input will make its way into the rest of the program and produce bugs.

Now, you can't always do this. There will be cases where you can't verify the correct properties right away, and you'll have to verify them during later processing. But you want as much verification to happen as early as possible so that you have your special cases around input logic centralized to one location as much as possible.

Use Schemas

Let's consider a function that parses some json.

def parse_student(text):
    try: 
        data = json.parse(text)
    except ValueError as error:
        raise ParseError(error)

    if not isinstance(data, dict):
        raise ParseError("Expected an object!")
    
    try:
        name = data['name']
    except KeyError:
        raise ParseError('Expected a name')

    if not isinstance(name, dict):
       raise ParseError("Expected an object for name")

    try:
        first = name['first']
    except KeyError:
        raise ParseError("Expected a first name")
    
    if not isinstance(first, basestring):
        raise ParseError("Expected first name to be a string")
    
    if first == '':
        raise ParseError("Expected non-empty first name")

That was a lot of work just to extract the first name, let alone any other attributes. We can make this a lot better if we can use a json-schema. See: http://json-schema.org/.

I can describe what my student object looks like:

{
    "type": "object",
    "properties": {
        "name": {
            "type": "object",
            "properties": {
                  "first" : {
                       "type" : "string"
                  }
            },
            "required": "first"
        },
    } 
    "required": ["name"]
}

When I parse I then do something like:

def parse_student(text):
    try: 
        data = json.parse(text)
    except ValueError as error:
        raise ParseError(error)

    try:
        validate(data, STUDENT_SCHEMA)
    except ValidationError as error:
        raise ParseError(error)

    first = data['name']['first']

Checking against the schema verifies most of the structure that I need. If the user input does not match the schema, the schema validator will produce a nice error message explaining exactly what was wrong. It will do so far more consistently and correctly then if I wrote the checking code by hand. Once the validation has been passed, I can just grab data out of the json object, because I know that it will have the correct structure.

Now, you probably aren't parsing JSON. But you may find that you can do something similar for your format that lets you reuse the basic validation logic across the different pieces of information that you fetch.

Answer 2

You can probably simplify your code by centralising the try/except validation of function args, and the conversion of exceptions to your exception class, into one or two decorators, which you apply to each of your methods and functions. google for python decorators for exceptions, and python decorators to validate args and you'll find stackoverflow examples like this and this.

You often don't need to explicitly validate method arguments as your code is going to cause exceptions naturally, and this might be good enough, as you cannot test for all eventualities.

Remember when writing a script, that often that your code will be even more useful if someone can include it as a library module, so make the main be specific to what a user would like from the command line, but in the library part don't try too hard to obscure where an exception stems from and so on.