We currently have 4 environments for a project (local/dev, test, acceptance, production). We created some integration tests which want to run. The big question we now have is where to run them?
Run them on all environments. The pro's for this is that we know for sure it works on all environments. The cons are that a) you don't want to stress your production environment with tests, b) Not all tests can be preformed on production since the external references from third parties of also production
Run them on single environment. So you pick one of the four. The problem with this is that environment specific settings (database, config) can cause bugs. So if you tests run fine on the test environment bug due to a config bug in production it breaks.
Create a production environment on the fly overwrite only the settings you really need for the test and run your tests on those. The pros of this approach is that for production you filter out the environment specific settings bugs as much as possible. They can still occur on the other environment, but the consequences are less. The cons are that this approach takes quite some time to set up, can it be skipped by minimized the differences between the environments..
I'm wondering, is there a standard for this? What would normally be done?