2

At my company, we've standardized on Bamboo for continuous integration. One of our products is a Server + a set of clients, and we want to test many different combinations. The way we do it is to have plans with two stages, the first starting the servers, and the second starting the clients and running tests. This is done to ensure that we've started the server before starting the client which connects to it, and we're using bamboo to choose which agent to run the server on.

Since a job cannot run across multiple stages, the first stage contains a bit of a hack - it actually starts the server 'in the background' so that the job 'finishes' without stopping the server, so that the server is still running during the second stage (we then have the client job stop the server after finishing the tests by sending a message to the agent running it). This already breaks the bamboo model, since bamboo doesn't know that the server is still running, and thus can no longer enforce that one thing is happening on a given agent at a given time, so many servers can be running on one agent at a time (this can cause problems, but it's not the main problem)

Worse, since a stage N+1 won't run unless all jobs in stage N were successful, those jobs 'cheat' in that they're always successful, even if they couldn't actually start the server. This is done because multiple combinations of clients & servers are being tested in each plan (I'm told we did it this way to limit the number of plans, I'm not sure why that's important over the number of jobs...)

In the end, this causes a lot of problems, especially since our bamboo system is rather congested - the first stage will quickly run, fail to start the servers (without any clear indication of this without someone looking at the logs, of a 'green' job nonetheless), while the jobs in the second stage will sit queued for a looong time, before just failing immediately for lack of a server, wasting time.

My question is, is there some better way of accomplishing this kind of thing with bamboo? It seems like the Bamboo model isn't really suited for it. The biggest reason for our problems is probably that we're relying on bamboo to choose which agents to run the servers on (otherwise we could just start the servers in the client job on-demand), but I'm not sure I can get that changed easily (or how to do that instead).

  • What exactly are you trying to test? Any decent integration/system testing would essentially be invalidated by all these hacks. – Robert Harvey Dec 2 '15 at 5:40
  • Testing that the client works with the server (we support both on various platforms, have different clients that talk the same protocol, have many ways of deploying a server, ect) – Bwmat Dec 2 '15 at 6:06
  • We have a ton of integration tests we run on each client. – Bwmat Dec 2 '15 at 6:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.