27

This is less a question about the nature of duck typing and more about staying pythonic, I suppose.

First of all - when dealing with dicts, in particular when the structure of the dict is fairly predictable and a given key is not typically present but sometimes is, I first think of two approaches:

if myKey in dict:
    do_some_work(dict[myKey])
else:
    pass

And of course Ye Olde 'forgiveness vs permission' approach.

try:
    do_some_work(dict[myKey])
except KeyError:
    pass

As a journeyman Python guy, I feel like I see the latter preferred a lot, which only feels odd I guess because in the Python docs try/excepts seem to be preferred when there is an actual mistake, as opposed to an, um… absence of success?

If the occasional dict does not have key in myDict, and it is known that it will not always have that key, is a try/except contextually misleading? This isn't a programming error, it's just a fact of the data - this dict just didn't have that particular key.

This seems particularly important when you look at the try/except/else syntax, which looks to be really useful when it comes to making sure that try isn't catching too many errors. You're able to do something like:

try:
    foo += bar
except TypeError:
    pass
else:
    return some_more_work(foo)

Isn't that going to lead to swallowing all kinds of weird errors that are probably the result of some bad code? The above code might just be preventing you from seeing that you're trying to add 2 + {} and you may never realize that some part of your code has gone horribly wrong. I don't suggest that we should Check All The Types, that's why it's Python and not JavaScript - but again with the context of try/except, it seems like it's supposed to catch the program doing something it shouldn't be doing, instead of enabling it to carry on.

I realize the above example is something of a straw-man argument, and is in fact intentionally bad. But given the pythonic creed of better to ask forgiveness than permission I can't help but feel like it begs the question of where the line in the sand actually is between correct application of if/else vs try/except is, in particular when you know what to expect out of the data you're working with.

I'm not even talking about speed concerns or best practice here, I'm just sort of quietly confused by the perceived Venn diagram of cases where it looks like it could go either way, but people err on the side of a try/except because 'someone somewhere said it was Pythonic'. Have I drawn the wrong conclusions about the application of this syntax?

1
  • I agree that the tendency towards try-except runs the risk of hiding errors. My opinion is that you should generally avoid try-except for conditions you expect to see (of course some exceptions apply to this rule). May 9, 2019 at 21:19

3 Answers 3

27

Use the get method instead:

some_dict.get(the_key, default_value)

...where default_value is the value returned if the_key is not in some_dict. If you omit default_value, None is returned if the key is missing.

In general, in Python, people tend to prefer try/except than checking something first - see the EAFP entry in the glossary. Note that many "test for membership" functions use exceptions behind the scenes.

5
  • Isn't this kind of the same problem though? If we're trying to work with a dict and doing work based on what's found, it feels like if myDict.get(a_key) is not None: do_work(myDict[a_key]) is wordier and yet another example of implicit looking before leaping -- plus it's slightly less readable IMHO. Perhaps I'm not understanding dict.get(a_key, a_default) in the right context, but it seems like it's a precursor to writing 'not-a-switch-statement if/elses' or magic values. I do like what this method does though, I'll look into it deeper.
    – user112358
    Jan 27, 2014 at 16:25
  • 1
    @Stick - you do: data = myDict.get(a_key, default), and either do_work will do the right thing when given default (eg. None) or you do if data: do_work(data). Only one lookup needed in either case. (It might be if data is not None: ..., in either case, it's still only one lookup and then your own logic can take over.)
    – detly
    Jan 27, 2014 at 22:25
  • 1
    So I have decided that dict.get() has pretty much saved me a ton of effort in my current project. It has reduced my line count, cleaned up a lot of terrible looking try/except/else junk and has just overall improved IMHO the readability of my code. I've learned a valuable lesson which I'd heard before but managed to neglect taking to heart: "If you feel like you're writing too much code, stop and look to the language features." Thanks!!
    – user112358
    Feb 3, 2014 at 20:09
  • @Stick - glad to hear it :) I like that advice, I've never heard it before.
    – detly
    Feb 3, 2014 at 22:32
  • 1
    Technically, which one is the fastest? Jan 24, 2019 at 14:16
4

Is missing key a rule or an exception?

From a pragmatic point of view, throwing/catching is much slower then checking if the key is in table. If missing key is a common occurrence - you should use condition.

Another thing in favor of condition is an else clause - it is much easier to understand when either block if executed, consider that even same exception class can be thrown from several statements in the try block.

Code in except clause should not be a part of the normal flaw (e.g. if key's not there, add it) - it should be handling the error.

3
  • 5
    "From a pragmatic point of view, throwing/catching is much slower then checking if the key is in table" — no it isn't. Not necessarily, anyway. Last I checked, the most common Python implementations do the try/catch version faster.
    – detly
    Jan 23, 2014 at 22:13
  • 2
    Throw/catch in python isn't all that big of a deal performance wise, to the degree that it is often used for flow-control in python. Additionally, there is the possibility in some situations where that dict could be modified in between the test and the access. Jan 24, 2014 at 0:17
  • @whatsisname - I thought of adding that to my answer, but TBH if you have race hazards like that, you have much bigger problems that EAFP vs LBYL :P
    – detly
    Jan 24, 2014 at 2:21
1

In the spirit of being "pythonic" and, specifically, duck typing - the try/except looks almost fragile to me.

All you really want is something with dict-like access, but __getitem__ can be overridden to do something not quite as you expect. For example, using defaultdict from the standard library:

In [1]: from collections import defaultdict

In [2]: foo = defaultdict(int)

In [3]: foo['a'] = 1

In [4]: foo['b']  # Someone did access checking with a try/except exactly as you have in the question
Out[4]: 0

In [5]: 'a' in foo
Out[5]: True

In [6]: 'b' in foo  # We never set it, but now it exists!
Out[6]: True

In [7]: 'c' in foo
Out[7]: False

Because the default return value is Falsy, access checking like that will work most of the time just fine.. It appears to keep the code simpler, too, which was why my co-workers and I had done it for months.

Unfortunately, a few months later, these extra keys actually ended up causing problems and hard-to-track-down bugs, because 0 became a valid value. And sometimes we would use in to check if a key existed, making the code do different things when it should have been identical.

The reason I suggest it looks broken is because in Python's spirit of duck-typing, all you should want is something dict-like, and a defaultdict fits that requirement perfectly, as would any object you might create that inherits from dict. You can't guarantee the caller isn't going to change its implementation later on down the line, so you should do the thing with the least side-effects.

Use the first version, if myKey in mydict.


Also, note that this is specifically about the example in the question. The python creed of "Better to ask forgiveness than permission" is largely meant to ensure you actually act correctly when you don't get what you want. For example, checking if a file exists and then trying to read from it - at least 3 things I can think of off the top of my head can go wrong:

  • A race condition is that the file might have been deleted/moved/etc
  • You don't have read permissions on the file
  • The file has been corrupted

The first one you can try to "ask for permission" on, but by nature of a race conditions it wouldn't necessarily always work; the second one is too easy to forget to check; and the third cannot be checked without trying to read the file. In any case, what you do after failing will almost certainly be the same, so in this example, it is better to "ask forgiveness" by using a try/except.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.