I have a simulation in Python which reads its configuration from a toml file. Since I have tons of parameters, the toml file can grow quite large.
This is an example file, similar in structure to my actual configuration file:
city.toml
name = 'New York'
[[houses]]
address = 'Foo Street 42'
color = 'red'
[[houses.residents]]
name = 'John'
age = 35
[[houses.residents]]
name = 'Mary'
age = 32
# [[houses]]
# etc ...
# [[houses.residents]]
# etc ...
Once loaded in Python, this turns into a dictionary similar to this:
city = {
'name': 'New York',
'houses': [
{
'address': 'Foo Street 42',
'color': 'red',
'residents': [
{
'name': 'John',
'age': 35,
},
{
'name': 'Mary',
'age': 32,
},
],
},
],
}
My issue is that the toml file can be quite repetitive. For example, if the user wants to simulate multiple residents identical to john = {'name': 'John', 'age': 32}
, they have to go to each line where a house is defined, and copy-paste the value a bunch of times:
[[houses]]
[[houses.residents]]
name = 'John'
age = 35
[[houses]]
[[houses.residents]]
name = 'John'
age = 35
[[houses]]
[[houses.residents]]
name = 'John'
age = 35
# [[houses]]
# etc ...
# [[houses.residents]]
# etc ...
which is both time-consuming and error prone. In particular, the user may run a simulation which is valid, but has undesired parameters, and only notice it later (if at all).
I'm thinking of solving this issue by "modularizing" the repeatable parameters. Something like:
# etc ...
[[houses]]
# etc ...
residents = ['John', 'Mary']
[[houses]]
# etc ...
residents = ['John']
[[houses]]
# etc ...
residents = ['John', 'Mary', 'James']
My idea is that, after parsing the main toml file, the Python code would be responsible for reading the strings in the residents
array and load the respective residents from individual toml files, like:
John.toml
name = 'John'
age = 35
Mary.toml
name = 'Mary'
age = 32
The final Python dictionary would then be constructed at runtime, merging different "submodules" of the configuration file.
My questions:
- Is this a good pattern to follow? Is this approach actually used anywhere?
- Are there significant drawbacks? (One I can think of is passing a value like
residents = ['Mark']
without an existingMark.toml
file - the code would have to deal with these situations somehow) - Are there any alternative solutions that I have not considered?