Continuous Integration using Docker

Question

One of the main advantages of Docker is the isolated environment it brings, and I want to leverage that advantage in my continuous integration workflow.

A "normal" CI workflow goes something like this:

• Poll repository for changes
• Pull from repository
• Install dependencies
• Run tests

In a Dockerized workflow, it would be something like this:

• Poll repository for changes
• Pull from repository
• Build docker image
• Run docker image as container
• Run tests
• Kill docker container

My problem is with the "run tests" step: since Docker is an isolated environment, intuitively I would like to treat it as one; this means the preferred method of communication are sockets. However, this only works well in certain situations (a webapp, for example). When testing different kind of services (for example, a background service that only communicated with a database), a different approach would be required.

What is the best way to approach this problem? Is it a problem with my application's design, and should I design it in a more TDD, service-oriented way that always listens on some socket? Or should I just give up on isolation, and do something like this:

• Poll repository for changes
• Pull from repository
• Build docker image
• Run docker image as container
• Open SSH session into container
• Run tests
• Kill docker container

SSH'ing into the container seems like an ugly solution to me, since it requires deep knowledge of the contents of the container, and thus break the isolation.

I would love to hear SO's different approaches to this problem.

Answer 1

You may invert the logic by moving test runner to the docker image:

1. Tests are run within the docker image, so they access the application like in any other context, and the presence of docker doesn't change anything.

2. Aggregating test results is where connection becomes an issue. The actual technology used will depend on your current infrastructure. It may as well be a message queue, a custom API or a direct access to the database.

3. Since it belongs to the process which runs inside a docker image to report the test results, and not to the central service to pull the results from the test runners, it means that again, the presence of docker image doesn't make any difference.

Answer 2

A fairly standard way to do testing with Docker images is:

1. Build a test image, based on the image you're testing, but adding the test runner and tests. (You don't have to do this; you could include all the testing in the base image, but it's cleaner to separate them).
2. Communication with Docker images is often done cleanly with filesystem overlays. So, mount a directory from the host into the container (e.g. with -v /path/to/workspace/:/tests, or something similar).
3. The test runner might output a result file, e.g. junit.xml or something like that. If it outputs into the /tests folder, then your CI environment can pick it up from there.
4. invoke the test runner by overriding the entrypoint on the docker run line. E.g. docker run container --entrypoint=/test-runner.py (or whatever is appropriate for your application). This works well for unit tests.

For integration tests, you usually have to orchestrate the spin-up of several containers outside of Docker anyway.