Is it safe to catch ImportError when trying to import optional modules?

Question

I usually see this pattern at least once in every Python project I work on. For example, in a Django project, this is often added at the bottom of the base settings file:

try:
  from .local_settings import *
except ImportError:
  pass

Also:

try:
  import simplejson as json
except ImportError:
  import json

This has always bothered me a little bit though; what if the module is imported successfully, but then it triggers an ImportError itself? E.g., in the first example, the local_settings module exists, but then local_settings tries to import a non-existent module.

Is this the safest way to import an optional module, is there a better way to achieve this functionality, or does it depend on the context/usage (and if so, what are the guidelines to deciding when to use this approach)?

Answer 1

It should generally be assumed that the optional dependency you are importing can work on its own. There is little difference between trying to import a missing optional dependency and one that cannot be imported because a required transitive dependency is missing.

In other words, why would your program need to care about simplejson not being available because it is not installed, or because a dependency of simplejson is not installed? Either way you cannot use simplejson.

For optional dependencies, it is up to the package installer to ensure that a dependency is correctly installed, including transitive dependencies.

For something like local_settings, there is indeed a (small) risk that a transitive ImportError is masked. You can always log the caught ImportError exception at level DEBUG or similar to make it easier to review what might have caused the exception:

try:
    from .local_settings import *
except ImportError:
    log.debug('local_settings failed to import', exc_info=True)

The logging package will include the exception information in the log for later inspection when you set exc_info to true.

Another option is to issue a warning with the warnings module, the standard library has a ImportWarning class for this:

import warnings

try:
    from .local_settings import *
except ImportError:
    warnings.warn('local_settings failed to import', ImportWarning)

Answer 2

This is generally safe, assuming your modules are free of import-time side effects.

If an imported module raises any exception (not just ImportError) at import time, it is removed from sys.modules. This means if any other code tries to import the module, it will not get a partially-initialized module. Instead, Python will try to load the module all over again, and presumably fail with an ImportError all over again.

This can cause issues if your modules have import-time side effects, since those effects are not undone. In particular, if the offending module successfully imports another module, the latter is not removed from sys.modules. This particular side effect is rarely a problem, but some side effects can be more troublesome.