I was talking with some coworkers about the application build and we have some divergences about what each one consider a good practice or something to worry about.

I learn that a good application build is one that is self sufficient, no matter how many dependencies and tests it have, the build will take care about them and run the build smooth.

Per example, if I have an application that uses Maven, I expect that a mvn clean install will work with the minimal requirements: Java, working Internet and maven installed (if you use mvnw in the project, even maven installed is optional).

In this scenario, not matter if your build run unit or integration tests with MongoDB, Oauth2, RabbitMQ... it's the programmer job to mock/stub these dependencies for the tests run and the build be independent from external services/databases/queues/etc running.

To be more clear, when I'm talking about integration tests, I'm talking about the the "narrow" ones: https://martinfowler.com/bliki/IntegrationTest.html, that typically run with the maven failsafe plugin in the Java world.

But some coworkers don't see any problem if an application build depends of a started MongoDB, Oauth2 server, RabbitMQ and etc running on the developers machine only for the build. Even using docker to help, for me this is the wrong approach, because:

  • The build is more slow.
  • Add complexity in the build process.
  • You need a Wiki/Guide only to explain how put the build to work.
  • Consumes more memory, because the external dependencies are not mocked.
  • You need to "reset" the external dependency each build.

When we are talking about the System Tests (or the "broad" integration tests), yes, we need all this external dependencies up and running, but this kinds of tests not occur during the build.

I think the resistant to make a self sufficient build cames from the complexity to deal with these dependencies on Integration Tests. Even using Spring, it's not easy to bypass a RabbitMQ and Oauth2 authentication, per example.

Although is clear to me what is the right approach, I can't find any discussions on Internet about this subject. What you guys think?

  • In principle I agree with you. Hopefully you've separated the part that directly interfaces with your external resources from your application logic. That would enable the type of build you want--and stub out the other items. – Berin Loritsch Jan 17 '18 at 21:32
  • Hi! This is easy with unit tests, but with integration tests (the narrow ones: martinfowler.com/bliki/IntegrationTest.html) these is harder to do, but possible using WireMock (for Rest calls), Embedded MongoDB, mocking Oauth2, etc. – Dherik Jan 17 '18 at 22:02
  • With all due respect to Fowler, writing a test that determines if a class calls a test-double properly is not an integration test, it's a unit test. All a test like that proves is that the class actually calls the methods that you asked it to. It's a completely uninteresting test from an integration perspective. The whole point of an integration test is to test interaction with real-life components, not interactions with dummy objects that don't contain any real functionality. I promise you, a real integration test will find bugs that "narrow" tests do not. – Robert Harvey Jan 18 '18 at 1:12
  • That said, whether you make the build depend on the successful passing of an integration test is entirely up to you. The integration test could be part of a QA effort, independent of the build. Your coworkers' position is probably that, if you're not going to break the build, then what is the point of running the test? – Robert Harvey Jan 18 '18 at 1:12
  • Robert, I will give you an example of integration test that is not a unit test, using TestDouble and the QA are not able to do. In a simple API, you, developer, make a test with rest calls to your app in a running tomcat, passing for all controller/service/repository, saving in a memory database (reseted after each test). If you have rest calls for another system, you Stub them (Test Double). When you say that integration test is about real life, this is real for the components of your application, not for the external ones (you won't test them in your application build). – Dherik Jan 18 '18 at 10:12

It is not uncommon to have tests that require external systems to be running, but I think it is uncommon to have a build that requires external systems to be running. That is an important distinction. Most teams have a continuous integration server that is used to put all code changes through their paces, but no developer should be required to run the entire suite of tests.

The Build

When a developer builds code locally, they should be able to kick off the build and end up with a reasonably tested binary:

  • All generated code is created
  • All code is compiled (including test code)
  • All unit tests are run
  • Finished package is generated

This build should be fast. The slower and more complicated the build, the less frequently it will be done. The more brittle the tests, the more likely they will be turned off or simply removed.

The integration tests are built, and ready to be run on demand, but they aren't run with the standard command-line build.

Continuous Integration (CI)

CI servers like Jenkins, TeamCity, AppVeyor, and the like are the tool that is used to constantly run the heavy handed tests. Your integration environment should be fully set up, with the CI server safely replacing parts of it.

This is where you would execute:

on every check-in.

The reason CI servers are necessary in this day and age is because of the complexity of a fully working environment. If your app was fully self-contained, then unit tests would be all that's needed.

That said, if someone breaks the build, your team needs a commitment that the fix will be found in 15 minutes or the change is rolled back... otherwise it's not continuous integration.

Dev Ops

The more you automate your build and deployment, the more likely the problems you find can be traced back to a code change. The more you can peer in to how your application runs in and integrated environment, the more informed your decisions can be for new features.

DevOps is a tool chain to allow you to do just that. Since your build/deployment/test/report chain should be done with proper continuous integration, the only thing left is instrumenting your application. By providing insight into how things are working, you can narrow down performance problems both in your integration environments and your production environments. The feedback from the instrumentation can help guide or improve requirements for your application.

  • Do you relegate integration testing to a post-build process of some sort, or do you dispense with it altogether? – Robert Harvey Jan 18 '18 at 1:15
  • @RobertHarvey, I'll do it before I put together a pull request, merge in code, or after I've rewritten a section of code to make sure nothing is broken. However, heavy integration tests are a major deterrent to doing it all the time. When managing build times you'll want as many quick running automated tests to run, saving the more complex ones for dedicated milestones. The CI server can handle all those other times more predictably. – Berin Loritsch Jan 18 '18 at 13:37
  • But we could understand that is a right moment to move the failsafe (integration) tests from the build (running together with unit test) to CI, right? If the project is small, does not make sense to me separate before the integration tests start to harm the build time. Per example, if you have a build that runs and 1 minute and with the IT the time is 2 minutes. By default maven execute the failsafe tests on build. – Dherik Jan 26 '18 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.