3

I have a code that reads a YAML file that consists of several parameters whose values are used throughout the code. For clarity, the YAML file has the following structure:

PARAM_1: value_1
PARAM_2: value_2
PARAM_N: value_N

These values are never changed after the YAML file is loaded into the program.

When the YAML file is loaded, it is directly converted to a dictionary params and then I access the values using the common notation params['PARAM_N']. The conversion is performed using the safe_load method in the yaml library.

I have discovered namedtuples after writing the code described above.

Namedtuples have several interesting features like the possibility to access the values using the dot notation and they are an immutable data structure as pointed out in other answers about this topic.

So, given the immutability of these parameters, should I convert the dictionary to a namedtuple (also to be more pythonic)? Or would it be useless? Or maybe do you see any drawback in the long run because of this conversion?

I think it would be a good idea, because the code would be safer (I cannot change a value by accident).

2
  • 1
    1. Changing a value by accident is a small risk if you scope your variables properly. Immutably is mostly useful when you are using concurrency. Nov 25, 2021 at 15:43
  • 5
    2. Whether you use dot notation or magic strings depends entirely on the nature of your use case. Which is more important to you: flexibility or strong typing? Nov 25, 2021 at 15:46

2 Answers 2

8

Defining a namedtuple or other type to represent the Yaml file can be a good idea compared to using a dict. But not because of immutability.

Python doesn't really support immutability the way C++ or Haskell does. It does support preventing further modifications to an existing object. E.g. fields of the built-in tuple cannot be reassigned. By extension, the same holds for namedtuple types. Dataclasses can be declared as frozen. But all of this is mostly just a runtime check. Do consider preventing reassignment of fields if that is a concern to you, but for many programs it's not actually relevant.

The bigger reason why you should consider introducing a type for the configuration data is that it makes the available fields explicit. With a dict, the user of this dict doesn't know which entries it is supposed to have or what types those entries have. With a class that represents this data, this information can be made explicit. And as an added benefit, you can get type-checking, autocomplete, and will get an error when you edit the config file but you mistype the name of a field by accident. So instead of

config = parse_file()
do_something(config['foo'], config['bar'])

it would be nice to have:

config = Configuration(**parse_file())
do_something(config.foo, config.bar)

Different ways to declare convenient types include the classic namedtuple() function, the new NamedTuple class notation, and dataclasses.

With the classic namedtuple function, you just list the fields:

from collections import namedtuple
Configuration = namedtuple('Configuration', ['foo', 'bar'])

Or equivalently with the newer typed syntax:

from typing import NamedTuple

class Configuration(NamedTuple):
  foo: int
  bar: str

Drawback: you can't directly validate that the configuration fields contain appropriate values. This could be alleviated by subclassing the namedtuple, except that because tuples are immutable their construction is special: you'd have to implement __new__ instead of __init__.

Dataclasses are typically more appropriate if you don't actually need the configuration to behave like an (ordered) tuple, though you can optionally have Python implement the corresponding methods as well. The class-based notation does expect you to provide either a field type or a default value though:

from dataclasses import dataclass

@dataclass(frozen=True)  # can prevent reassignment if required
class Configuration:
  foo: int
  bar = ""

Validation could be done with a __post_init__ magic method.

In any case, it's probably better to do validation outside of the class:

import yaml

def get_config() -> Configuration:
  with open('config.yaml') as config_file:
    raw_config = yaml.load(config_file)
  if not isinstance(raw_config, dict):
    raise TypeError('config file must be a mapping')

  foo = raw_config.pop('foo')
  if not isinstance(foo, int) and int >= 0:
    raise TypeError(f'entry `foo` must be a nonnegative int but was {foo!r}')

  bar = raw_config.pop('bar', "")
  if not isinstance(bar, str):
    raise TypeError(f'entry `bar` must be a string but was {bar!r}')

  if raw_config:
    raise TypeError(f'config file contains unknown entries: {list(raw_config)}')

  return Configuration(foo=foo, bar=bar)
3
  • Beautiful and very detailed answer! I really did not think about making the available field explicit, thanks! What is your take about using the class types.SimpleNamespace and doing SimpleNamespace(**params_dict)?
    – blunova
    Nov 25, 2021 at 17:25
  • 1
    @blunova The SimpleNamespace type just provides object-like access to a dict. This is a purely syntactic change. All the points about dicts with respect to type checking, autocomplete, and typo detection apply.
    – amon
    Nov 25, 2021 at 17:33
  • 1
    @blunova from the docs: SimpleNamespace may be useful as a replacement for class NS: pass. However, for a structured record type use namedtuple() instead. Nov 26, 2021 at 7:21
1

When using named tuples, you create a tuple in PARAM_N[value_N]. If you know the PARAM_N before you run your code, then you can use named tuples. Otherwise, staying in the dictionary is inevitable.

More specifically, if you know exactly what parameters the YAML file will have before the run (e.g. name, age, address) , then you can use named tuples. If you are not sure what will you receive, then you are stuck with the dictionary. It's the easiest way to check for the existence of a key without creating duplicate data.

4
  • 1
    It's not really true that you have to use a dictionary if you don't know the keys beforehand, since you can dynamically create namedtuple classes. Something like Config = namedtuple('Config', list(config_dict.keys()) Not saying it's necessarily a good idea.
    – Jasmijn
    Nov 25, 2021 at 15:49
  • @Jasmijn Then you are creating double data; a list with the parameters and the named tuples. list_of_parameters=[PARAM_N] and PARAM_N(value_N). I don't find this a good practice. Also thanks @RobertHarvey . I haven't drink coffee yet. Nov 25, 2021 at 15:59
  • I'm not sure what you mean by "double data" or what you mean by list_of_parameters or the notation PARAM_N(value_N).
    – Jasmijn
    Nov 25, 2021 at 17:28
  • I mean to have the "same" data multiply times in different variables. To have a list_of_parameters (for the names of parameters) AND the named tuples (PARAM_1(value_1),PARAM_2(value_2),..., PARAM_N(value_N) type). Also, something new. A new module called frozendict from Marco Sulla . It creates an immutable dictionary. Maybe this helps. Nov 25, 2021 at 19:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.