Use Exceptions for control flow in order to increase performance?

Question

I add a lot of elements to a list of lists. If the list of lists my element should be saved to, does not exist yet, I handle this by catching an exception and adding a new list to my list of lists.

The reason I do this instead of checking every time, if the specified list already exists, is, that I add thousands of elements, to just a few lists. So the case, that the list was not created yet, occurs only a few times. Thus I think it is faster to just raise an exception and handle it, instead of checking list lengths thousands of times.

Here is an example of what I mean:

try:
    self.layers[pos].extend(node)
except IndexError:
    self.layers.append([])
    self.layers[pos].extend(nodes)

(I know the index pos can not be higher than the length of layers. Thus appending one if an error occurs is always enough)

So back to my question, is that bad coding practice, or is it okay to do it like this for the sake of performance?

I found this discussion about it, but performance was not a topic there.

Answer 1

Yes this is a bad coding practice, since it obfuscates what is going on.

Worse, since throwing an exception is such an expensive operation in pretty much every language, this won't even get you anything performance-wise! (Though I would measure to make sure)

Answer 2

Using exceptions for flow control is highly frowned upon in most languages, but not in Python. Using exceptions for flow control in Python is "pythonic." Exceptions are at the heart of how python for loops work, which terminate the iteration on receiving a StopIteration exception. Python has a slightly derogatory term for languages such as C++, Java, and C# where programmers are strongly advised to *never* use exceptions for flow control. These are "Look Before You Leap" (LBYL) languages.

Python on the other hand is an EAFP language: It's Easier to Ask for Forgiveness than Permission. Using exceptions for flow control is at the very heart of the language. Your code is nicely pythonic.

That said, if you find yourself repeating yourself in multiple places, you might want to follow the DRY (don't repeat yourself) principle and create a list of lists (LoL) (or perhaps a defaultlist class and thereby avoid my dry sense of humor) that extends the builtin list class. You'll want to override the __setitem__ method (and possibly the __getitem__ method) so that that missing elements are filled in with an empty list. Then you can append at will, and you won't even have to use a try / except block. Now your code will follow the "Nike" principle: Just do it.

Answer 3

Python has a builtin method setdefault which does exactly what you need:

self.layers.setdefault("pos", []).extend(node)

Answer 4

I hear this misguided idea a lot: "exceptions are situations in your code that you should never reach. By their name, they suggest exceptional, unexpected and uncontrollable situations. Throwing an exception should stop the execution of your code all together."

This is misguided because, suppose you want to access a file, and the file cannot be accessed because it is locked, so a "File is locked" exception is thrown; what are you going to do? Stop the program? If you are writing a GUI application, the only reasonable thing to do is to display a "retry, cancel" dialog, meaning catching the exception and taking action on it.

The quote is actually dangerous because it may prompt the novice programmer to do something like if (not resource_is_locked) access_resource which will of course appear to work, but will fail under seemingly random circumstances under race conditions.

The only part of the above quote that is true is that exceptions suggest exceptional situations. From the moment you catch the exception it becomes an expected situation, and from the moment you put any code in the catch block it is a controlled situation.

So, it is perfectly fine to define a situation as "exceptional" and handle it with an exception. I would not call it flow control. I would just call it exception handling.

Just be sure to a) first exhaust all other possibilities of doing it elegantly without exceptions, and b) if you must do it using exceptions, then clearly document what is going on.

Also keep in mind that in Windows with Dotnet, throwing an exception is terribly expensive due to the twisted way microsoft has implemented things. In Java, either in Windows or in Linux, throwing and catching an exception is not terribly expensive. (Though still more expensive than it ought to be, in my opinion.)

Answer 5

First of all, as long as you don't know that the piece of code in question actually is a performance bottleneck, don't bother optimizing it and write it in the most clear, straightforward way that you can think of.

---

Whether the exception is faster than the explicit check depends entirely on the language/compiler. To be more precise, there are two points involved:

1. To throw the exception, the machine code must encode an implicit check. If you encode an explicit check yourself, that duplicates the amount of checks that are done. This should indeed slow down your code.

2. Throwing an exception is exceptionally expensive in many languages. It may require a huge amount of avoided double checks to make up for a single exception that's thrown. This may easily make your optimized code run significantly slower.

It depends entirely on your compiler whether it is able to optimize away the implicit check controlling the exception if you write an explicit check for the same condition into your code. So, if you compiler is smart enough, the explicit check will always be faster. However, if your compiler is stupid but handles exception in a quick fashion, the implicit exception based check is faster.

As such, truth is only in measurement. Implement both versions, run them through a realistic performance test, measure their execution times, and compare. Anything else is just guesswork.