Using a scripting language in place of a config file looks great at first glance: you have the full power of that language available and can simply eval()
or import
it. In practice, there are a few gotchas:
It is a programming language, which means:
Itit is a programming language, which needs to be learnt. To edit the config, you need to know this language sufficiently well. Configuration files typically have a simpler format that is more difficult to get wrong.
Theit is a programming language, which means that the config can get difficult to debug. With a normal config file you look at it and see what values are provided for each property. With a script, you potentially need to execute it first to see the values.
Itit is a programming language, which makes it difficult to maintain a clear separation between the configuration and the actual program. Sometimes you do want this kind of extensibility, but at that point you are probably rather looking for a real plugin system.
Theit is a programming language, which means that the config can do anything that the programming language can do. So either you are using a sandbox solution which negates much of the flexibility of the language, or you are placing high trust in the config author.
So using a script for configuration is likely OK if the audience of your tool is developers, e.g. Sphinx config or the setup.py in Python projects. Other programs with executable configuration are shells like Bash, and editors like Vim.
Using a programming language for configuration is necessary if the config contains many conditional sections, or if it provides callbacks/plugins. Using a script directly instead of eval()-ing some config field tends to be more debuggable (think of the stack traces and line numbers!).
Directly using a programming language may also be a good idea if your config is so repetitive that you are writing scripts to autogenerate the config. But perhaps a better data model for the config could remove the need for such explicit configuration? For example, it may be helpful if the config file can contain placeholders that you later expand. Another feature sometimes seen is multiple config files with different precedence that can override each other, though that introduces some problems of its own.
In the majority of cases, INI files, Java property files, or YAML documents are much better suited for configuration. For complex data models, XML may also be applicable. As you've noted, JSON has some aspects that make it unsuitable as a human-editable configuration file, although it is a fine data exchange format.