4

I am trying to package a Python module for pip, following the guide here.

One area I would like feedback on is best practices or convention for making my module configurable. The module is a library for talking to federated RESTful services / local datastores, and we use it both as a command-line library as well as a Django app. So I would like it to be manually configurable as well as configurable from Django's settings.py file.

Our current method of doing this is horrible -- it relies on a set of settings.py files inside of the library, which are overwritten whenever we pull from git, not changeable during runtime, etc.

My idea for a solution is to wrap the entire library in its own class, and in the __init__ method, do something like:

try:
    from django.conf import settings
    self._remote_host = settings.REMOTE_HOST
except:
    self._remote_host = DEFAULT_HOST

But that only seems to take care of Django and seems cumbersome -- what if someone wants to use our library with Flask, or something else? Is there a more universal way to make a Python module configurable by external tools plus have a default? Or is this a lost cause, and I should stick to configuration on init via arguments?

3

There are two problems come together here:

  1. Your package is a singleton although there exist many different configurations.
  2. Where to store the data and how to migrate - which database to use.

Singleton

I did a fun project with lots of modules which needed configuration themselves. The project was intended to run once per computer.

The alternative is to put everything into classes and instantiate them with a configuration. Tist would allow many different configurations to exist within one program. When you may have this need you should restructure your whole code.

Database

Maybe your project is for a company and shall be developed for a longer time and the configuration shall not be thrown away. Then you may need to keep in mind many previous configurations when changing the default values and updating the configuration. Proper databases have solved that problem.

My tradeoff solution

In my case

  • The configuration can be thrown away if my model changes.
  • The package runs not only once per process but also once per computer.
  • Only one thread accesses the configuration.

There is a module called config.py with the following methods:

import config
config.load()
config.save()

You use it like this (Example1):

config.load()
config.my_value = 'test'
config.save()

There is also a file called constants.py which should better be called default_config.py. It has the functions

import constants
constants.default_configuration()
constants.config_file_name() # where to store the config data

And for (Example1) the constants.py should look like this:

def default_configuration():
    return {'my_value' : 'default'} # to avoid attribute errors

The config module saves and loads the configuration using pickle. If no configuration is found for a variable the default configuration is used.

You will need a coding style that always fetches the configuration from the config.py. It shall not be stored in a local variable or attribute over a longer time since it can change.

My previous version is called runningConfiguration. It has no explicit load and save and also no default attributes.

2

TL;DR: Pass as args in init. Balance the use of libraries against the limiting factor they may represent to others who don't use those libraries and don't want to.

There is a general rule of thumb that there is an inverse correlation between number libraries a project uses (especially those that dictate policy like config or db) and the population of people that can/want to use your library.

For example, if you make a library that only work with Django, you are limiting yourself to only Django users. Same with config, specific database, etc.

Dependent libraries that don't dictate policy are not as impactful, but that doesn't mean there is no impact. Reasons around not wanting to use these kinds of dependent libraries will probably run the gambit:

  • minimizing program size (rightly or wrongly this can be a reason)
  • security issues
  • compliance issues
  • have to use another dependent library that conflicts
  • have to use a different version of the dependent library
  • etc

For these reasons I would suggest you always go with the minimal dependence on other libraries. In this case that would mean passing the args in init and allowing the consumer of your library to decide how and which config they want to use.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.