Dealing with settings that will change on a per-environment basis?

Question

We have a bunch of appSettings in our ASP.NET web application, not to mention connection strings, that change on a per-environment basis. Not just for different environments to which the web application is deployed, but across different developer machines too. Therefore, changing these with a Web.config transform isn't good enough because these transforms aren't carried out when developers debug the application.

At the moment the settings are hard-coded into Web.config but if a developer wants to change them during development, this obviously means the change is flagged up in source control which is undesirable. These changes shouldn't be checked into source control.

How can we move these settings out so that they can be changed without affecting source control? In the past I have worked on projects where we used configSource to point to an external config file which was then excluded from source control, and a .config.example file was put in its place which had to be copied to the .config file. The trouble with this is that by default, it gives a broken build because this config file is missing until it's created. We want a working build that can be pushed to VSTS for deployment. How can we do this while keeping the per-environment settings separate from source control?

Answer 1

@amon's answer is spot on. The best way to deal with this in a tech stack agnostic way would be with build scripts, etc. to create the files you need at build time.

However, since you are working in .Net land, you could look into NConfig. It's a NuGet package that makes it really easy to load up machine specific configs. (There are also ways to use environment variables, but I've never done that.) You could still commit machine specific configs to source control (or ignore them with your SCM's ignore file), but even with them committed, it won't hose up someone else's build.

Answer 2

The way I've done it is through configSource for machine-specific config files (appSettings, mailSettings etc.) as this was the most compatible and appropriate way for ASP.NET in my eyes.

I excluded these files from source control and took a leaf from Symfony 4 development by creating .dist versions of each configSource file for source control, so that there is a 'reference' version sent with each version of the source code (e.g AppSettings.config.dist).

In order to fix files not being created on build, you need to set the Build Action to Content. (Right-click, properties, Build Action is under 'Advanced').

This seemed to be the best practice for us as it gives a config reference whilst still making it the environment owners responsibility to create them.

This also provides a separation between changing config files and static values found in your Web.config.

You could then automate the creation of config files from the .dist files in VSTS via a script or if you're feeling really spicy a function in Startup.cs to create these files if they don't exist.

---

As as aside - when pushing out builds you can provide overrides/transforms for config files by simply naming them whatever stage of the build you're in (such as 'Debug' or 'Release'), more info can be found here.

This can be incredibly useful in certain situations (especially for constant values in your Web.config).

Answer 3

It is not the job of the application to know the configuration for each and every environment. Instead, the environment should provide its configuration. That might happen through config files or environment variables, or any other mechanism.

If you were to provide a default configuration file, this config will always be wrong in some case. I once worked on a project where the default config was set up for a test environment. One day a colleague walked up to my desk: “Could you perhaps change your database connection? When you run your tests, this overwrites changes in my database instance.” Oops. They had committed their configuration, and I didn't know I had to change the defaults.

If the config were for a production environment, we have the additional difficulty that it might contain plaintext passwords (or other secrets). Even when you trust everyone who has access to the source, this is a bad practice and introduces unnecessary risk. For example, the risk of accidentally using parts of the production environment when running tests.

The only winning option is not to play. Do not provide a default config.

Instead, it is part of the build/deployment process to supply the correct configuration. You say “The trouble with this is that by default, it gives a broken build because this config file is missing until it's created. We want a working build that can be pushed to VSTS for deployment.” No. Part of the build is a pre-build script that creates the configuration file. This might be as simple as copy config.buildServer config. It is unreasonable to expect the code to run without any environment-specific steps.

Answer 4

So a 'standard' setup might be:

• web.config file with settings for local dev
• git for source control
• build and version on TeamCity https://www.jetbrains.com/teamcity/
• package build with OctoPack
• publish package to internal nuget feed
• deploy package with octopus https://octopus.com/
• override web.config settings in octopus with environment scoped variables.

Obviously there are alternate options for each step, but this gives you a build that you can work on locally, versioned builds which can be deployed to environments and environment specific config settings which are outside source control.