2

I encountered a situation where the dev team suggested that they will push bug fixes to our dev environment rather than to the test environment, and they asked QA to test there first. I am not sure if this is a good practice, I always thought it is better to test in the test environment, but maybe there are circumstances where testing in dev environment makes more sense?

  • 3
    Every place has their own definition of what "the dev environment" and "the test environment" are. There's no way for us to know, from just the content in your question, whether testing in your "the dev environment" is appropriate. – Eric King Dec 2 '18 at 18:19
  • Re:"...test there first". Do you mean that QA will do a 'confidence' test in the development system first but still do the 'official' test on the production system; OR do you mean, QA will only test the changes on the development system and claim success based on that alone? – Dunk Dec 6 '18 at 15:03
1

General Advice

The closer the environment is to a clone of production, the more accurate your testing will be. The further away it is, the more caveats.

Many development environments are not similar to production. Mostly because they are playgrounds for developers to trial different configurations, software bundles, services, etc... This doesn't mean they are useless (for testing), they just aren't going to be definitive in proving that something has been fixed.

For those lucky few that have high-quality prod-like development environments, then they really are no different to test environments, go forth and test. Rebuild the environment first though.

For those environments with some drift from being production like, then run a sub-set of the tests. These tests aim to show sufficient quality to get that code into a more production-like environment. In the more production like environment I would re-run those tests again.

State of Development in your Organisation

The fact that your developers are suggesting this raises a question in my head. Why is it this feature/fix not getting into the Test environment?

  • Is there a way to reliably duplicate the defect?
  • Is the defect replication automated?
  • Is it clear to the developer what the defect is?
  • Is the developer collaboratively testing and exploring the issue?
  • Is there some sort of penalty for getting a defect fix wrong when it has been deployed to Test?
  • Is there some sort of penalty for treating a defect like any other feature?
  • Is the Test environment occupied by a release that is slated for before/after when this defect should be released?
  • Is it hard for the developer to obtain diagnostics from the Test environment?
  • Is the Test environment less production like than the development environment in this case?
  • Does the defect fix require changes to the environment that cannot be applied to the Test environment until all the future release use that environment?

Figuring out the root issue will help you make changes to the processes, technology, culture, and the system the bug fixes are for.

For clarity: A Penalty is any negative consequence that occurs as a result of an action. So when I say "is there a penalty?" I am talking about specific responses that may be spelled out (the number of defect/fixes directly controls remuneration), or implied (the team member is mocked).

4

Well why do you separate dev and test environments?

Normally it's because testers want control over when their environment changes. It's annoying when a test that was passing starts failing without you noticing because you didn't realize the environment changed.

Now a dev team pushing to a dev environment is not weird at all. It's not weird for the dev team to run tests in dev. It is a bit weird to have built a test env for the QA team and then have the QA team use the dev env to test.

But none of this is a problem so long as no one goes and changes the environment when people aren't expecting it to change. Managing change is the only reason to separate these anyway. So, if for some reason, they want to suspend that separation, maybe to tighten the time between bug fix and QA feedback, and maybe just this once, then I think it could be fine. Just don't surprise anyone with this.

If it works really well for you it may change your way of testing.

2

There is no right answer for this question. This depends on how your team develop new features, how they are delivered on the production, the development cycle of the team, if the project have budget to have many environments, etc.

I already work on a software where only some developed features are delivered to tests. To not merge a lot of branches that could not be delivered to production so soon, we decided to have a test environment separated having only the code that will be delivered on the next version of the software. So, use the dev environment it is not the best solution in this example.

For general cases, the better solution is have a test environment separated, because also it's normal that the development team make some changes on the software that makes the environment unstable for a certain period. When this happens, the QA team will not be affected, because they will receive only in your environment more stable builds.

But test in the dev environment can work too, but needs more discipline and maturity from the dev and QA team. If the dev team have the concern to maintain the development environment always stable, when something goes wrong they must prioritize the stabilization of the environment for the QA team continues their tests. The QA team, of course, needs to be aware about these kind of blocks, report and understand that this can happen sometimes.

1

It would have been common sense that you ask the developer if there was a particular reason to test in the dev environment. If there is a good reason you test in the dev environment. If the devs say there is a good reason but you don’t see the reason, you have the choice between testing in the dev environment and possibly starting a fight. If the devs admit there is no good reason, then you test it wherever you think is right.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.