13

We are a small-ish firm with multiple teams who manage their own git repositories. This is a web platform and each team's artifacts are deployed at the end of the day for nightly tests. We are trying to formalise the process around versioning and packaging.

Every team has a master branch where they do day-to-day development. Quality assurance members of each team want the artifacts from their team's changes deployed into a test bed where all the components are combined by chef. Artifacts are tarballs but I would like to convert them into RPMs so we can think and reason about versions correctly.

The release process involves cutting off a release branch from the development branch (master in most cases) of each the git repositories. This is then given to quality assurance who run tests and sign-off on a set of artifacts.

For eg this is a typical git repository with its associated release branches:

 0-0-0-0-0-0-0-0-0-0 (master)
   |           |
   0           0
 (rel-1)       |
               0
            (rel-2)

I am stuck trying to figure out a scheme to perform versioning of packages coming from development branches. We don't want to excessively tag the master branch of each repo and restrict tags to release branches only. But we should be able to query the deployed packages in the test machines using standard yum/rpm semantics. What would development versions look like when the master branch has no tags? I understand that git describe can provide me a useful representation of a build version but that works well when various release points on the branch are tagged.

EDIT1: In response to @Urban48's answer

I figured I should explain our release process a little more. For purposes of this discussion let's assume we have branch master in all repositories. The master branch is considered the development branch and is deployed to an automated CI-CD enabled QA environment. This is where a subset of nightly tests run to ensure the stability of master. We look at this pipeline of jobs before cutting a release branch. Our release branches are short lived. Say, after cutting a release branch (from a stable master), a full regression is run, fixes are made and deployed to production. This takes about a week to do. We release almost every two weeks to production.

Our feature branches are always cut from master and undergo some amount of developer testing before merging with master upon which they undergo the CI-CD stability checks.

Hotfixes are made on hotfix branches (cut from release branches) and deployed with minimal impact testing into production.

Our versioning strategy for release and hotfix branches follows semver. Release branches during the QA cycle go through versions like v2.0.0-rc1, v2.0.0-rc2 and finally after QA sign-off become v2.0.0.

We sometimes do dotted releases for small features which are merged to release branches (and then to master) where the versions become v2.1.0. And hotfixes assume the v2.1.1 pattern.

The question however, is not about versioning these branches. I would prefer not to change this versioning scheme altogether. The only change comes about for development branch ie. master. How can I indicate reliably in the CI-CD environment which version exists wrt the previous release into production. This would ideally be done through smart git tagging but something that does not excessively tag the master branch is preferred.

  • why not add -rc.<build_number> to the builds from development branches, and once released from master/release branch just use a x.y.z? – Urban48 Nov 22 '16 at 17:07
  • What would precede the rc suffix? That would dictate the major.minor development version. rc and build number can be obtained based only on that. Also rc on master doesn't make sense because we never release from master. We tag our release candidates today on release branches as parts of the release cycle – tsps Nov 23 '16 at 4:37
  • i see, then what about not releasing from multiple branches, but cherry-pick complete features to a single release branch, where you can then use tags. versioning packages(rpm or deb) will become easier, everything that is not on the release branch will have an rc suffix. – Urban48 Nov 23 '16 at 8:27
3
+50

Hmm, well I have a .net example which might be technology agnostic.

I'll just do a quick summary.

  • git repo per component with gitflow branching strategy

  • all commits to develop trigger a team city build

  • teamcity build amends the version with manual major minor + the build number in AssemblyInfo.cs ie 1.1.hotfix.build

  • teamcity triggers nuget packaging using the same version number for nuget published libraries

  • octopus deploys finished build to qa for manual testing (assuming all tests pass)

  • if everything is good, manually deploy version to production via octopus.

Now this does mean you get A LOT of versioned packages floating about. We did experiment with use of the -prerelease flag but this required a further manual move package from prelease to 'normal' step and a requirement to rebuild components which depended on them.

The key thing is to version every build uniquely via some central process.

Then you are in the situation of 'hmm what versions do i want' rather than 'omg, what version have i got??'

Edit: re comments.

Just to underline that key thing really. Decide what branch constitutes completed software and version ALL commits to it. only deploy versioned software from this branch.

My view though is that you need to address your branching strategy.

  • don't version feature (or any dev) branches.
  • do version master
  • only use packages from master
  • 1
    I have explored git-flow multiple times in the past and unfortunately unable to push it through. Changing the branching strategy and any git workflows of developers is a hard problem to solve. We want to change versioning strategy so the numbers make sense when deployed on development, test and production. – tsps Nov 23 '16 at 2:57
  • Also, we do have versions on the development branches today. It's just that we have to tag every commit, sometimes twice to version this way. I would like to provide a cleaner view of development by indicating that the current version going out is vnext+builds similar to how git describe – tsps Nov 23 '16 at 3:00
  • well you can still build and version commits to master. do they not even use feature branches? – Ewan Nov 23 '16 at 7:26
  • if you dont want to version master. then you have to manually update the minor version on the release branch. which means manually configuring your builds everytime you make a new release bran h – Ewan Nov 23 '16 at 7:30
  • Feature branches exist and I need to apply versions to them too. Not averse to adding tags/versions on master, just that it is excessively done today. Any strategy that points to the point at which I can tag the master branch and indicate that it is a development version as opposed to versions on release branches is helpful – tsps Nov 23 '16 at 9:30
1

Let me offer alternative workflow, that may solve your versioning problem, or just help you think of more ways towards the solution.

A bit of philosophy first..
(I'm making few assumptions about your workflow, please correct me if I'm wrong)

  1. Tests are run retroactively:

    Branch out of maser to feature branch, then turn it into an artifact for testing by QA.
    the problem is: what if the tests fail, then master may be broken!

    side effects:

    • It make the developers not trust master

    • Since you branch out from master to release branches, what happens if previous release is broken? it stops your new feature integration until master is fixed again.

    • when bug is fixed in the release branch, merge back to master may create merge conflicts. (and we don't like conflicts)

  2. small code merges and fixes:

    Its hard to keep track of all code that is merged to master, like small fixes or changes that are not part of a certain feature.

    side effects:

    • developer is not sure if he should marge this small fix now or later.

    • should i version this new small fix as well?

    • At any point of time its not clear what is the state of master branch, and what code is floating in there

    • something broke the build, that is not part of the new feature. and its really hard to track where it came from

My idea for git work flow is as follows:

enter image description here

branch out of master for new feature development like you do already. but now instead of releasing from this newly created branch, have this feature cherry-picked and merged into the release branch.

Now you have better control over what is going into a certain release.
Now its really easy to isolate development versions and stable versions.

You can keep on building artifacts from those feature branches for QA.
If everything is ok, merge that feature back to master, and cherry-pick this feature/bug-fix/hot-fix into the release branch.

versioning
The feature branches version can use some name convention like 1.2.3-rc.12345

the the versions in the release branch will only use 1.2.3 (1.2.3 > 1.2.3-rc.12345 also one less thing to worry about)

This workflow fixes the problems mentioned above, and more.
It also propose sane versioning strategy, and most of this release cycle can be automated.

I hope it will help you in some way, i'll gladly discuss any edge cases you can come up with.

p.s:
I apologize for my English, its not my main language.

  • Thanks for your detailed answer. I will respond in the original post as EDITs since the comment bar does not allow for enough text. – tsps Nov 24 '16 at 3:07
0

Your process seems very convoluted and getting a bit number of tags seems unavoidable.

Two ideas come to mind:

  1. You treat the "master" branch as what we typically have into the "develop" branch. This pollutes the message branch a lot.

  2. When the QA finish with their job you can have the build/CI server to create a report (eg. A text file) with the tags of all the repos in use. Now you will have a file with the versions which can be posted to the repo. Then you tag the branch with the release version only and if you want to check the versions of they individual components you can check the report.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.