Building a plugins-based code in Python

Question

I have a program which perform different actions depending on the plugins that are passed. For example, python main.py -m foo -m bar will perform the actions of foo and bar.

The structure of the directory is:

├── Dockerfile
├── README.md
├── docker-compose.yml
├── entrypoint.sh
├── modules
│   └── foo.py
│   └── bar.py
├── main.py
├── requirements.txt
└── settings.yaml

And in main.py, I have a function for each of the plugins. Assume the following:

def foo():
    ...

def bar():
    ...

if args.all or 'foo' in args.modules:
    foo()
elif args.all or 'bar' in args.modules:
    bar()

This works but there must be certainly a better way to work with plugins , because as the number of module grows, having a function and an if for each one doesn't look like a good option.

What's the recommended way to do an implementation like this one?

Answer 1

If you can place some requirements on the modules, then you can make it quite dynamic. The requirements that you must place on the modules are at least

1. The name of the file must match the name passed in the -m argument
2. The file must be placed in the modules/ folder
3. The module must support the functions that the main application will want to call on it. These functions will have a name and arguments as defined by the main application (so independent of the module).

With these requirements, you can use the python module importlib to dynamically load your modules as python modules and call functions on them.
For the --all option, you can just iterate over the files in the modules/ folder.

Answer 2

An alternative to multiple if-statements could be a dispatch table:

modules = {
  'foo': foo,
  'bar': bar,
}

if args.all:
  args.modules = modules.keys()

for module_name in args.modules:
  try:
    module = modules[module_name]
  except KeyError:
    ...
  module()

Some command line argument parsers such as Click have built-in support for running multiple commands, so you don't have to write this dispatching logic yourself.