2

This is how my team has it's development environment set up:

  1. Each team member has a local development environment on their own computer, where they check out the code from a shared repository.

  2. Every developer work in their local development environment, once they finished writing the code they commit back.

  3. The code is then checked out to a staging/testing server with the same environment setup

  4. If everything is OK, it will be pushed to a production server.

We're trying to integrate unit testing and BDD (behavior driven development) into this setup and we'd like to use a continuous integration server (Jenkins) too.

My question is where should we install this and where should we test the code?

  1. Install the full testing stack for each developer: TDD tools + BDD tools + Jenkins?
  2. Install the test tools only on the staging/testing server and only test code here once each developer made his/her code commit?

How is your setup?

Improve this question
6
  • Try programmers. Anyway, steps #1/2/3 seem to indicate a lack of proper SCM in general (and a long write-to-test turn around cycle).
    – pst
    Commented Jan 19, 2012 at 22:28
  • Why is lack of proper SCM in your perspective? How's your SCM setup?
    – feketegy
    Commented Jan 19, 2012 at 22:34
  • Does user acceptance testing happen somewhere in this process?
    – Matthew Flynn
    Commented Jan 19, 2012 at 23:34
  • @pst, The OP seems to have listed some key points as a background for his question, so I can't really see how you can conclude the lack of a proper SCM and long turn-around based solely on the OP's points above. In any case, how does your comment help to improve the OPs question, or even help to answer it?
    – S.Robins
    Commented Jan 20, 2012 at 6:19
  • @MatthewFlynn: How does your comment help to improve the OPs question, or even help to answer it?
    – S.Robins
    Commented Jan 20, 2012 at 6:22

4 Answers 4

Reset to default
3

Installing a CI (Jenkins in your case) on every single workstation would be an overkill. This thing is pretty heavy! Common practice is to have it only on one server and make it check for modifications (commits) in the repository every X minutes (so it automatically builds) - or you can make a post-merge hook in your VCS to trigger a build.

What do you mean by testing tools? Every dev should be able to run unit/integration tests from his environment if that is what you are asking about. In fact it is a good practice to run tests before you commit (I know that the CI is there to check it anywhere but still I prefere not to crash the builds).

The setup at all the companies I worked was pretty similar:

  1. everyone has its own local dev env where he can run unit/integration tests if he wants to
  2. there is obviously a common VCS repository where we all commit
  3. there is a CI on a server somewhere polling for new commits from the VCS repo and if it finds one it starts a CI job building the whole project checking for any failures (build or test ones).
  4. we had a nightly build for each project which would build the application from the latest sources from the repository and deploy it to the server in the middle of the night when no one is using it.

You can go even further and try to adopt continuous deployment. We do not have that at my current job since the procedures of deploying the application are very corporation-like.

Nice things to have:

  • mails incoming after every build (saying it either succeeded or failed, always send a mail, if you want not to send one after a successful build it is a mistake since you don't know if the build was triggered or not)
  • a screen in a visible to all/most developers place with the current state of all your builds
Improve this answer
4
  • After several years of development with a continuous integration server, we found that the important mails were: (a) A mail for each commit where the build becomes broken. (b) A mail when the build is still broken. (c) A mail when the build is fixed. That is, if commit 10 breaks the build, commit 11 has an unrelated new feature added (but the full build is still broken), and commit 12 fixes the build, then commit 10 gives a "The build has become broken" mail, commit 11 gives a "The build is still broken" mail, and commit 12 gives a "The build is working again" mail.
    – Trevor Powell
    Commented Jan 19, 2012 at 23:44
  • We also found that it made a surprisingly huge difference if broken-build mails contained the generated build errors. And if your CI server is fast enough to keep up with all the commits, then including the name of the person who broke the build made a mammoth difference in how often builds became broken in the first place, since nobody wanted their name to be sent in one of those mails.
    – Trevor Powell
    Commented Jan 19, 2012 at 23:47
  • But, without a mail about a successful build how did you know that the CI job actually got triggered? What if the server was down? Oh and I indeed forgot to mention the stacktrace and the name of the person who committed the code which broke the build, yeah.
    – Zenzen
    Commented Jan 19, 2012 at 23:58
  • Early on, we had some projects which were concerned about that, and for those, we had the CI server send out a "Build still good" mail after 'n' number of consecutive successful commits (typically 20). But later on we made build status screens as you mention in your post, and those would have made it really clear if the CI server had ever stopped picking up new commits for any reason.
    – Trevor Powell
    Commented Jan 20, 2012 at 0:10
7

You shouldn't need the entire stack for each developer. Having a Jenkins server locally would be overkill and isn't necessary.

Better instead to have one system as a build server, which runs a commonly used build script, and to take care of all the messy deployment stuff that you don't want happening ad-hoc at each workstation.

Each programmer has their development environment, the full suite of testing tools and APIs required, and can run the common build script locally. Doing this allows all of the builds to work in the same way, and to allow through the process of continuous integration to determine if their changes are likely to break the build. We have a commitment where I work of making sure that when our code is moved to the version control system, that it should not break the build, so building locally allows us to catch any silly problems that could leave the build in a seemingly permanent broken state.

Once we've checked our code in, the build system confirms that the source code has change, and it automatically launches a build process. If the build works on my PC, but breaks on the build system, then it is usually because either I forgot to check something in, or because another developer has changed something in the meantime that might have affected the way my code executes... although this usually happens to be because I would have forgotten to refresh my code base prior to my build, which (yes I know) should probably happen automatically at the start of my local build script execution.

Improve this answer
1

If you're really doing TDD, the developers are writing and running tests locally, so you definitely need the testing tools on each developer machine.

Improve this answer
1
  • The idea of a build server is to be separate from any individual developers work space and in a clean room environment. This ensures reproducability.
    – user1249
    Commented Jan 22, 2012 at 11:14
1

Our setup:

Each developer has a their own dev server (vm)

  • so all developers can develop on the same version of the same OS, with the same set of system packages installed
  • so there isn't 1 shared def platform to make sure system updates or toolchain updates don't impact everyone until they have been vetted.
  • to support homogenous development vms, but heterogenous development tools (whatever dev tools make each developer most productive)
  • which is on the same secured network (accessible only through VPN/2-factor auth) as the large (shared) systems we integrate with
  • to be on the same OS version and configuration as the platform that software will be deployed to in production

Each developer does TDD, but writes automated acceptance tests, not unit tests

  • assures testing of integration points
  • trades off some assurance in component correctness for overall system correctness
  • requires less time to do large refactors because we don't pay for test updates on refactors

Each developer has a toolchain, and the codebase. The test suite is part of the codebase.

  • to reduce bug vectors by assuring that everyone is working with the same versions

CI server is created the same way as a dev server, with the same dev environment. It runs with every commit, executes tests, tags code and produces installable packages

  • to assure the test suite is run against every commit
  • to assure every successful build is tagged and packaged since it is considered an RC
  • to assure all RCs are built the exact same way every time
  • to assure that packages are created the same way every time

Packages are installed in QA for more extensive testing

  • our tests significantly reduces, but doesn't completely eliminate the need for manual testing

The same packages that pass QA are installed in Production

  • to reduce bug vectors by using the exact same bits that were already extensively tested
Improve this answer
2
  • Please could you explain how each developer having their own VM dev server solves the problem of "integrating with large systems that are not available on the desktop network"? Do you install fakes of those large systems on the VM? Also, you say "to manage the infrastructure packages separately from the requirements of the developers desktop applications" - are these "infrastructure packages" you speak of the same thing as the "large systems" you mention?
    – Jayy
    Commented Aug 12, 2015 at 15:15
  • @Jodes updated online, but basically the VMs are on a secured network that have network access to the large (shared) systems.
    – snakehiss
    Commented Aug 12, 2015 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.