The built-in logging module of python 3.x allows for 3 ways to define a custom logger:

  1. INI-formatted file
  2. dict, json, yaml
  3. python (directly in code)

In my opinion, it is easier to define a custom logger's config directly in the code, because this does not require anyone to understand other formats (it is not obvious what the dependencies are in INI, yaml, and json formats). If config is written in python, then any coder can understand how the Formatter, Handler, and Logger relate to each other programmatically, if not conceptually.

Are there reasons why it is better to define the logging config in a separate config file? I'm guessing this would have to do with wanting multiple deploys with different logging levels, but I can't imagine a compelling use case for this.

  • 4
    The most important advantage to a configuration file is it's not code. – Robert Harvey Jul 29 '19 at 22:16
  • 4
    It's fairly typical to have a lot more logging in a Dev/Integration environment compared with a LIve/Production environment. Your live environment would generally be stable so you usually wouldn't want to be filling production logs with loads of debug and trace output. However it's very likely that you'd want to know exactly what's going on with unstable builds in a development environment. – Ben Cottrell Jul 29 '19 at 22:35
  • 2
    In addition, if you have strange behavior in production you might want to switch to a higher logging detail level temporarily without rebuilding the application with different settings. – Hans-Martin Mosner Jul 30 '19 at 4:46

It's the difference between things which can never be changed by the user and things which are configurable.

If you gave out your software as compiled binaries on cds for people to install on their home computers. Each user would want a different logging configuration. If the config is hardcoded they can't change it.

The same thing applies when it's your software and its deployed to servers. You want that division of labour so that you can adjust the config without changing the code when you deployment environment changes. maybe you want to log to a different drive, or log differently in your test environment to live, or you run out of disk space and need to turn logging off etc etc

  • This is absolutely true, when distributing compiled binaries. When distributing source code the configuration file solution needs to be easier to understand than the source code solution to make this worth it. Remember, configuration files might not have source code in them but they are still a data language and many end users aren't coders. You make a real good point about understanding who's using this. Capture the case when you aren't distributing binaries. – candied_orange Aug 3 '19 at 10:35
  • some people call that documentation. you should make a pdf, print it out, bind it and deliver hard copies to your users with the cd – Ewan Aug 3 '19 at 11:12
  • Ick, no. No one reads documentation. People barely read comments. What's needed is discoverable files, expressed in an understandable language (data or code) and unique key words so googlers can gather around blog posts that explain the 42 bugs you never imagined were going to be involved in solving this problem. – candied_orange Aug 3 '19 at 11:26
  • Anyway, I'd like to upvote your answer. Could you please expand it to cover the case where what's being distributed isn't compiled binaries? – candied_orange Aug 3 '19 at 15:36
  • im not sure i would change the answer for that situation. would you even distribute the config? maybe several examples with lots of comments – Ewan Aug 3 '19 at 18:21

I used to do it in code and switched to yaml. Now I am going back to code.

1) this is too much boilerplate:

from logging.config import dictConfig
import yaml
import os
import logging
log = logging.getLogger()

As opposed to:

from logcon import log

2) The import mechanism is hierarchical

It will automatically import a python file from the first available place in the pythonpath. So you can have a logcon.py in your site-packages and it is automatically used as a default.

3) The yaml file has no error handling

Loggers and some formatters are defined by a module. If that module is not present then your program fails. It even fails if you have no handler for that logger. So the only way to remove a logger is remove all references. With a python file you could wrap that in try/except and ignore the error. Then you can reuse the same config more easily.

4) The python file is much more flexible

You can read in a config; set settings depending on the environment etc..

5) It is just as easy to change as a config file

The only people changing your config file are programmers. They can change a .py file as easily as a .yaml file. You can still have different configurations. Furthermore logging.yaml files have code in them anyway by having classes included. It is just less flexible.

Of course you can also mix and match. Have a .py file with exception handling and all the python features; and that reads a .yaml with log levels.

  • 2
    I've had plenty of non programmers changing logging configurations. If you want to make a point about what we're putting them through show us an example logging.yaml file. – candied_orange Aug 3 '19 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.