0

In Python projects, if your project spans multiple Python modules and you need to pass some data between the modules, as far as I know, there's 2 main ways:

  • Pass the data as function arguments
  • Create a module that stores all your global variables, and import the module from the modules that need to use it

When does it make sense to use a global module for storing your variables, as opposed to passing the data as function arguments? When you have too many different modules using the same data, all needing it to be passed to them?

8
  • 3
    The answer is essentially never. It might help to know more about your use case, but global state is almost never a good idea from a software engineering point of view. Oct 31, 2021 at 13:24
  • 1
    @VincentSavard Django uses it for django.conf.settings. Seems never is a pretty bold statement, essentially implying that one of the biggest Python projects out there is doing it wrong
    – John
    Oct 31, 2021 at 13:43
  • 1
    From what I can tell, django.conf.settings is a Singleton. It seems to be designed as a way to provide configuration, and it seems to be intended to be used in an immutable way (essentially, a list of constants), which is quite different from a global variable as mentioned in your question. Regardless, there is also a lot of literature on why singletons are not recommended. Oct 31, 2021 at 14:22
  • 2
    The way your question is phrased,the only way I can see this going is a big argument about whether there ever might be any case in which global data module make sense or whether it truly never is a good idea. From your previous comment it sounds to me you already have decided where you stand. I think a way more useful way to ask would be to bring up a specific case where you think passing data isn't working for you, so we can have a discussion about concrete details and not an abstract what if scenario
    – Helena
    Oct 31, 2021 at 14:32
  • This scenario comes up frequently when I'm programming. I'm not looking for a specific example to be analyzed, or to start an argument. I'm looking to figure out a general idea of when I should use one, or the other. For example, @VincentSavard's comment suggests not using a globals module unless you have immutable data, which, if others agree, might be a good guideline. I've seen some code where function A passes an arg to function B, and B does nothing with it except pass it to function C. I'm curious if that's a good reason to use a global module instead
    – John
    Oct 31, 2021 at 15:09

1 Answer 1

8

Once upon a time in the early days of computers, there were no function parameters or return values. There were only global variables. This was an utter mess and completely unmaintainable. It is really difficult to track how data flows through such a “spaghetti code” program that uses global variables to communicate, because every part of the code could change everything.

Python is a very flexible languages, so it is possible to reach into another module and change its global variables (especially since there are no true constants in Python). But with great power comes great responsibility. It is rarely appropriate to do this. Legitimate examples I have seen include:

  • monkey-patching for tests
  • initializing and configuring singletons
  • manipulating process-global data such as sys.env

In the comments, you mention django.conf.settings. Since these settings are by design process-global (you can only have one Django app running per process) it's OK to access and modify those settings. However, if I had designed Django, I might have chosen another approach for configuring settings that does not rely on global variables.

For software that you are designing, it is probably better to make your data flows explicit and to pass data via function parameters and return values. If you have a lot of related data, you can create a class to group these variables. With Python 3.7 dataclasses, it is now very easy to create such classes. I find that this helps to write well-structured and easy to test code: all the data dependencies are explicit, which helps to figure out what has to be provided in a test case. If a function depends on a lot of unrelated stuff, this could indicate that the code has been structured in an awkward way.

For example, consider this code where we have some business logic to transform data and save it in a database:

DB_CONNECTION = ...
DATA = ...

def my_function() -> None:
  # some tricky business logic
  DATA['y'] += DATA['x']**2
  DB_CONNECTION.execute('INSERT INTO table VALUES(?, ?)', (DATA['x'], DATA['y']))

# caller:
DB_CONNECTION = ...
DATA = ...
my_function()

A caller of this function must make sure that the DB_CONNECTION and DATA variables are initialized before calling the function. Such “temporal dependencies” are bad API design in most cases. Here, we have also introduced an artificial constraint that there can only be one connection and one data in the program.

We could refactor this by turning those variables into function arguments. This makes it clear to the caller what has to be provided for the function to work. It also makes it much more straightforward to test the function:

def my_function(data: Data, *, conn: Connection) -> None:
  # some tricky business logic
  data['y'] += data['x']**2
  conn.execute('INSERT INTO table VALUES(?, ?)', (data['x'], data['y']))

# caller:
my_function(..., conn=...)

But why is this function taking both data and a database connection? For testing the tricky business logic, we would still have to provide a database connection and then read the data back from the database. In some cases, this can be simplified by splitting this code into a “pure” part that just manipulates our data, and a part that performs external interaction.

def my_inplace_transformation(data: Data) -> None:
  # some tricky business logic
  data['y'] += data['x']**2

# pure alternative that does not modify data:
def my_transformation(data: Data) -> Data:
  return { 'y': data['x']**2, **data }

def save_data(data: Data, *, conn: Connection) -> None:
  conn.execute('INSERT INTO table VALUES(?, ?)', (data['x'], data['y']))

# caller:
data = ...
my_inplace_transformation(data)
# alternative: data = my_transformation(data)
save_data(data, conn=...)

Here, the caller has regained a bit of responsibility: it's now the caller's job to pass the data between the functions, in particular to call the save_data() function if the data shall be saved. But now the main business logic is nicely isolated and very easy to test.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.