1

I have a list of application configs and business configs in a particular root folder inside our git repository.

Below is our current structure as of now:

  1. We have a root Data folder and inside that we have multiple sub-folders which has bunch of application and business conigs.
  2. Each sub-folder has different number of application or business configs compared to other sub-folders.
  3. Total sub-folders can be more than 10 and total number of files inside each sub-folders are random and they can be 100 max.
  4. Each files are very small like few KB's. Each Sub-folder and file names are different than what I have shown below.

I came up with below names just to make my question clear to understand -

Data
    folder1
        files1.json
        files2.json
        files3.json
        files4.json
    folder2
        files5.json
        files6.json
        files7.json
    folder3
        files8.json
        files9.json
        files10.json
        files11.json
    folder4
        files12.json
        files13.json
        files14.json
        files15.json
    folder5
        files16.json
        files17.json
        files18.json
        files19.json
    folder6
        files20.json
        files21.json
        files22.json
        files23.json

Now with each commit we make a zip folder of all the files and subfolders exactly in the same structure as shown above so once you unzip it on the disk, it will show root folder Data, each sub-folders inside root folder and all files inside each sub-folder exactly as show above. For example: Modifying any file above or adding a new file in any subfolder or adding a new folder, we make a zip folder of all the files and subfolders exactly in the same structure as shown above.

Problem Statement:

Now we have a new requrirement where we need files for each environment (dev/stage/prod). Below is my requirement:

  • I can have some files for each environment with different content. For eg: file1 can be in dev, stage and prod with different contents.
  • But there can be some files which are same for all 3 environment so don't need to duplicate those files for each environment.

How can I represent my folder structure and files in my git repo so that it is easier for me to understand by looking at the structure and also achieve above those two new requirements in a clean way?

Below is one design I was able to come up with but it violates my second requirement as I mentioned above since I am duplicating each file in different environments even they are same.

Data
    dev
        folder1
            files1.json
            files2.json
            files3.json
            files4.json
        folder2
            files5.json
            files6.json
            files7.json
        folder3
            files8.json
            files9.json
            files10.json
            files11.json
        folder4
            files12.json
            files13.json
            files14.json
            files15.json
        folder5
            files16.json
            files17.json
            files18.json
            files19.json
        folder6
            files20.json
            files21.json
            files22.json
            files23.json

    stage
        folder1
            files1.json
            files2.json
            files3.json
            files4.json
        folder2
            files5.json
            files6.json
            files7.json
        folder3
            files8.json
            files9.json
            files10.json
            files11.json
        folder4
            files12.json
            files13.json
            files14.json
            files15.json
        folder5
            files16.json
            files17.json
            files18.json
            files19.json
        folder6
            files20.json
            files21.json
            files22.json
            files23.json


    prod
        folder1
            files1.json
            files2.json
            files3.json
            files4.json
        folder2
            files5.json
            files6.json
            files7.json
        folder3
            files8.json
            files9.json
            files10.json
            files11.json
        folder4
            files12.json
            files13.json
            files14.json
            files15.json
        folder5
            files16.json
            files17.json
            files18.json
            files19.json
        folder6
            files20.json
            files21.json
            files22.json
            files23.json

1 Answer 1

3

Dev/stage/prod is just one case of site-specific configuration, which most software that I'm familiar with handles using application defaults and site overrides.

You could have top folders application for application default configuration and dev, stage, prod for environment specific overrides. The override folders would only contain config values that differ from the default.

I don't know why your configuration is dispersed over so many files, it's hard to judge this structure without knowing how it is used by the application, but it looks somewhat over-engineered. It's reasonable to keep configuration options in one place when they tend to be changed together. This would suggest that environment specific config parameters such as service addresses, replication factors, resource limits etc. should go into just one file, which makes the environment config folders pretty trivial.

2
  • so you mean to say I can have application folder which will have default configuration and override folders for each environment like dev, stage and prod which will have configs that are different for specific environments?
    – cs98
    Aug 6, 2020 at 5:19
  • 1
    @cs98 yes, that's the idea. One thing you may also consider is that some configuration values such as production database credentials should never go into the application repository. You might store them in a tightly controlled credentials repository, or better yet use a credential store, and only store a sample environment specific config file in git. Aug 6, 2020 at 6:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.