Background info and what I've tried
Within each of two mostly distinct environments, say 'Research' and 'Production', there is a need for a structured parameter file. The file contains things like database connections, tables, and some other parameters, nothing unusual.
The current approach is to have two distinct parameter files, say "params.txt" and "params_prod.txt" (for production). (Also, in case it helps, we're not actually using .txt for this, we're using a markup language, but the details of that don't really matter for this question.)
Most of the information is duplicated between these two files, leading to lots of copy/paste. Plus, in version control, everyone has to manually make changes to two files and we have to trust that they are propagating changes if checking things in.
Needless to say, this leads to headaches to resolve when some divergence happens between the files. We have tried workarounds, like writing tests to check that the files are identical in all the ways they need to be, but this is not a perfect science since the structure of the files can change and the definition of what, exactly, must be identical between them changes too.
I have had one idea:
Just use a single file, but create sub-sections within the file. There can be a sub-section for general parameters that are always shared, another section for things that should be treated as the 'default' parameters (the 'Research' environment settings) and then another section for the 'Production' environment settings.
We already have code that parses the parameter files and instantiates objects, loads data, etc. etc., based on the parameters. We could go back and add some options to that code, such that it loads data according to which sub-section of the parameter file it is told to use and ignores parameters from the other section.
This has some benefits: (1) everything is in one file, so there's no "just trusting" that people are propagating changes. (2) It also does not require copy/paste code since anything that is shared between both environments needs to only appear in one section at the top of one parameter file. (3) If anything, this should make the parameter files themselves more modular and easier to use. (4) We save time/cost that would have been spent creating complicated test-based work-arounds that check whether the files are being propagated together during check-ins with version control. We don't need to do that in this case.
It does have costs too: (1) time spent making the newly formatted parameter file specifications in XSD so we can validate it and ensure it is backwards compatible; (2) costs if we do need to re-code and make our software-that-interfaces-with-parameter-files have options for whether to use 'Research' or 'Production'. (3) If any of the properties that are currently considered to be in the sub-section that is shared, but which suddenly become things where we want to have different options between 'Research' and 'Production', we would then have to refactor all parameter files to move that item down into the other sections.
(3) Seems like the biggest worry, but it also forces us to constantly re-factor these parameter files, which is a good thing in my view. Besides, if we want flexibility about re-assigning a parameter from one sub-section to another, there ought to be better ways to achieve it than duplicating the entire file.
Question
Are there any significant pitfalls that I am failing to realize about the proposed idea regarding don't-repeat-yourself vs. build-in-flexibility trade-offs in parameter file design?
