I have an Amazon ECS service that relies on configuration files that contain business rules. The business rules are changed periodically by non-programmers. Total size of the rules files is approximately ~100K. Currently we store them in S3 and create a new folder every time anything changes.

I feel that we can manage the business rules better if we put them in a real source control like Git. They would take less space, we would be able to keep history of who changed what when, and we would be able to keep multiple sets of rules in branches. This means, however, that we will have to host a Git repo somewhere and clone it into every container.

The parameters we want to optimize for are:

  • Reliability. We don't want our system to be down, because GitHub or GitLab decided to do some maintenance or suffer from a DDOS attack.
  • Cost. We want new system to be cheaper than storing historical files in S3, or at least have comparable cost.
  • Startup time. We don't want new system to cause longer startup time than downloading things from S3.

Hosting git repo on Amazon CodeCommit and using shallow clone (--depth=1) to copy files over seems like a good solution, but maybe I am missing some important pitfalls.

  • Do you need to tie the maintainance of the rules to the deployment of the rules? Why not continue to use s3 for the current production version, and git for the development/historical version. And then release from git to s3 any time theres a new released version. Commented Jun 26 at 11:34
  • 2
    "we would be able to keep history of who changed what when" only if you expose git to those clients/non-programmers. Which is a bad idea. Git as an implementation detail sounds ok, not as exposed service. Once you couple yourself with it, and people start using it, it will be hard to replace it later to something else. I would start with clearly defining requirements first (including authentication) and then building API definition around it. Chosing implementation goes afterwards.
    – freakish
    Commented Jun 26 at 13:54
  • 1
    It seems that you're only concerned with raw technical costs, but what about the cost of peoples' time maintaining those rules, and the cost of identifying and fixing any mistakes/mishaps? I would expect these would be where the real potential for cost-saving would be found. (Also, you describe the people maintaining these rules as "non-programmers" -- except the fact they do this job means they literally are programmers, just presumably without the training, experience, discipline, etc - a very risky situation with potentially very expensive outcomes if they break things). Commented Jun 27 at 7:11
  • @BenCottrell really has the relevant metric here: human labor. If cost is an issue, reduce human labor first. Storage is cheap by comparison. Commented Jun 28 at 13:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.