I have designed an evaluation tool (in python) and need some help to make it more user friendly. The tool requires ~100 (nested) parameters, which it gets from a yaml file and stores internally as dictionaries (e.g. one dictionary for data loading, another for preprocessing, ...). However, it is really difficult for the user to know which parameters are necessary due to the following two reasons:
- The input parameters change due to new feature requests. This sometimes breaks backwards compatibility for old yaml files.
- Depending on which kind of evaluation shall be performed and which input data is used, some parameters are either optional or mandatory. There are interdependencies.
Currently, the tool simply crashes mid-run if a necessary parameter is missing. Users then often have a hard time to figure out why the parameters are wrong (they may also suspect that the tool is broken). How can this be improved upon?
My solution ideas
I have the following ideas of my own but feel like there might be better approaches / architectures / design patterns out there.
- Define a default parameter file, which is then overwritten by user parameters. Thereby, new params can be added without breaking backwards compatibility (already done).
- Create reference yaml files, which are always tested in CI. However, lots of combinations are possible and the user would still have to manually check the reference yaml files...
- Introduce a parameter-checking method at the beginning. Maybe via converting the dictionary to a set of named tuples which have the benefit of being immutable and throwing an error directly at the beginning instead of mid-run.
- Your ideas...?
Thanks in advance!