1

We currently have four environments in AWS, development, test, beta and production. Previously, we released from develop to beta at the end of each sprint (2 weeks). We would create feature branches off of develop.

We would then deploy to beta and production every month by merging across environments. Such as merging beta into production, test into beta. This process was both slow to deliver value, and high-risk as lots of changes by different teams would build-up.

So I've attempted to alter the process so we branch off of master, and create pull requests into each environment instead. With the intention of releasing smaller deliverables, more often.

However we're now in a position where we're dealing with misaligned branches, changes getting out of sync. Being unable to pull any change other than master into our feature branches, otherwise we 'pollute' our feature branches with other teams changes.

My initial intention was for us to deploy things to beta/production as soon as they're ready, but the business is insisting on two week cycles still. Which means changes are building up again.

So it's led me to despair, surely it's possible to just deploy small sets of changes across several environments? I just wondered if anyone has a solution to this, or any advice at all?

  • How large is your company? How many developers? – Robert Harvey Mar 28 at 14:58
  • @RobertHarvey - 35 engineers, across UK and India. It's a tech/ai company, 100+ staff overall. Very product/feature driven – Ewan Valentine Mar 28 at 15:27
  • Your process strikes me as a bit heavyweight for such a small company. I agree with Evan's answer below. – Robert Harvey Mar 28 at 15:30
2

Don't make branches to match your environments.

Build from dev and deploy to dev for testing. Merge into master at the end of a sprint.

Build from master and deploy those releases to beta and production

-1

Speed and Crashing

The business is probably hesitant about deploying to production quickly, because the defect rate is high/higher than they desire (probably somewhere around zero), and the cost/effects of this are higher than they desire (also probably somewhere around zero).

So if you want quick releases to production, then you need to show the business that the number of expected defects will be very very low, and their cost/effects will also be very very low.

A Plan to Move Safely at Speed

The only known way that might achieve this is to invest in your automated testing, your production monitoring, and finally the ability to quickly and safely roll back a broken deployment automatically based on that monitoring.

This is almost impossible to achieve without a healthy creative and diligent culture shared within and between the teams responsible for development and support.

Evidence

If you must acquire an alpha environment to demonstrate the culmination of the culture + automation. Collect the data and prove that the tests + monitoring + automated roll back both ensure a very low defect to deployment rate, but also a robust and accurate recovery for those defects that do get through.

To put this into perspective, those supporting the system should have nothing to do at all during a deployment, except pressing the deploy button.

Where to start

So many places, you know the system well, what is it that keeps causing pain?

  • Are the defects getting in because of bad coding style? How about a static analyser?
  • Are the modules fine, but don't work together? How about integration tests, or a fuzzer?
  • Are simple mistakes continually happening? Code reviews...
  • Is the feature not what was asked for? UAT, and tighter co-operation with users

I'm pretty sure there is a laundry list of problems around somewhere, if not higher someone new, and listen to what they point out as weird. They might have a point.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.