When should I use a global variable module in Python?

Question

In Python projects, if your project spans multiple Python modules and you need to pass some data between the modules, as far as I know, there's 2 main ways:

• Pass the data as function arguments
• Create a module that stores all your global variables, and import the module from the modules that need to use it

When does it make sense to use a global module for storing your variables, as opposed to passing the data as function arguments? When you have too many different modules using the same data, all needing it to be passed to them?

Answer 1

Once upon a time in the early days of computers, there were no function parameters or return values. There were only global variables. This was an utter mess and completely unmaintainable. It is really difficult to track how data flows through such a “spaghetti code” program that uses global variables to communicate, because every part of the code could change everything.

Python is a very flexible languages, so it is possible to reach into another module and change its global variables (especially since there are no true constants in Python). But with great power comes great responsibility. It is rarely appropriate to do this. Legitimate examples I have seen include:

• monkey-patching for tests
• initializing and configuring singletons
• manipulating process-global data such as sys.env

In the comments, you mention django.conf.settings. Since these settings are by design process-global (you can only have one Django app running per process) it's OK to access and modify those settings. However, if I had designed Django, I might have chosen another approach for configuring settings that does not rely on global variables.

For software that you are designing, it is probably better to make your data flows explicit and to pass data via function parameters and return values. If you have a lot of related data, you can create a class to group these variables. With Python 3.7 dataclasses, it is now very easy to create such classes. I find that this helps to write well-structured and easy to test code: all the data dependencies are explicit, which helps to figure out what has to be provided in a test case. If a function depends on a lot of unrelated stuff, this could indicate that the code has been structured in an awkward way.

For example, consider this code where we have some business logic to transform data and save it in a database:

DB_CONNECTION = ...
DATA = ...

def my_function() -> None:
  # some tricky business logic
  DATA['y'] += DATA['x']**2
  DB_CONNECTION.execute('INSERT INTO table VALUES(?, ?)', (DATA['x'], DATA['y']))

# caller:
DB_CONNECTION = ...
DATA = ...
my_function()

A caller of this function must make sure that the DB_CONNECTION and DATA variables are initialized before calling the function. Such “temporal dependencies” are bad API design in most cases. Here, we have also introduced an artificial constraint that there can only be one connection and one data in the program.

We could refactor this by turning those variables into function arguments. This makes it clear to the caller what has to be provided for the function to work. It also makes it much more straightforward to test the function:

def my_function(data: Data, *, conn: Connection) -> None:
  # some tricky business logic
  data['y'] += data['x']**2
  conn.execute('INSERT INTO table VALUES(?, ?)', (data['x'], data['y']))

# caller:
my_function(..., conn=...)

But why is this function taking both data and a database connection? For testing the tricky business logic, we would still have to provide a database connection and then read the data back from the database. In some cases, this can be simplified by splitting this code into a “pure” part that just manipulates our data, and a part that performs external interaction.

def my_inplace_transformation(data: Data) -> None:
  # some tricky business logic
  data['y'] += data['x']**2

# pure alternative that does not modify data:
def my_transformation(data: Data) -> Data:
  return { 'y': data['x']**2, **data }

def save_data(data: Data, *, conn: Connection) -> None:
  conn.execute('INSERT INTO table VALUES(?, ?)', (data['x'], data['y']))

# caller:
data = ...
my_inplace_transformation(data)
# alternative: data = my_transformation(data)
save_data(data, conn=...)

Here, the caller has regained a bit of responsibility: it's now the caller's job to pass the data between the functions, in particular to call the save_data() function if the data shall be saved. But now the main business logic is nicely isolated and very easy to test.