I've recently watched Integration Tests are a Scam by J. B. Rainsberger and am now looking for more material on the subject. I have to say, I'm shocked by how much we're doing wrong, (i.e. integration testing when we should unit test), intrigued by the concepts described by Rainsberger but also confused about how to apply them. I would like to have more of the described collaboration tests and contract tests but I don't know where to start.

The only things that got stuck in my mind are the 4 questions the tests need to ask:

Side A:

Do I ask the right question?
Can I deal with the answer?

Side B:

Can I answer a question?
Do I answer correctly?

But how do I apply this to some random method in my application stack?

Is there a book or a tutorial or example out there that takes a real world example and applies these ideas of isolated micro tests? Ideally the example uses Java, Spring + PowerMock / Mockito / EasyMoock

Any literature that deals with these concepts in general and helps me understand them better would be appreciated.

Also if there are forums out there where I can ask more detailed questions about how to go about correctly unit testing and maybe even refactoring existing code and post examples would be nice.


Edit - some additional thoughts:

Is there a general rule, when to use a mock and when a stub to isolate a class under test from it's collaborators? Can it be applied to the 4 questions?

The best mocking frameworks seems to be PowerMock, allowing me to precicely define for each test what class I want to have mocked and what it should return or is there anything better that you have used to ask the above questions? Are there good tutorials out there that use PowerMock to apply some or all of the given principles to some part of a real world application stack, say a DAO or a GUI?

Edit 2 - an example

Just some example of the kind of methods I want to test and my thoughts..

I have a web service that saves orders. At this stage we're not too worried about the best security, so to have some, the service will also take a username and password to authenticate the save request. Once authenticated, the OrderManager is called to save the Order. Internally the manager decides if its a new order so it has to be created or an existing one that has to be updated. (That shouldn't matter to the WebSerice, right?)

public class OrderService {
    private AuthenticationManager authenticationManager;

    private OrderManager orderManager;

    public void save(String username, String password, Order order) {
        authenticationManager.authenticate(username, password);
        try {
        } finally {

Now I am wondering: What exactly am I testing here? I'm thinking there should be tests for authentication success and failure and for order saving success and failure.

But how can I split that up into the 4 questions? My class under Test is obviously OrderService (OS) and the collaborators are OrderManager (OM) and AuthenticationManager. (AM) So I have the following tests, please correct me here, I am just thinking aloud:

OS <--> OM

  • OS asks OM to save an order (what kind of different parameters do I test here? null and Order? Does it matter if Order is correctly initialized?)
  • OM answers a save call by calling some other internal method, I I test if that method gets invoked?!
  • OS should not fail if OM does not fail
  • OS should fail if OM fails

... What else?

And then of course OS <--> AM:

  • OS asks AM to authenticate - I guess I test how the AM reacts to different kinds of username / password?
  • ...

Now my first conclusion:

As far as the WebSerice is concerned I can only test 2 out of 4 questions: Side A. Now I have to look at the OrderManager and AuthenticationManager and see if they can answer the questions of side B. Right?

Secondly - database access:

Authentication and Order persisting obviously requires some data in the database in a production environment. For my unit tests however I won't need them so I'll just mock the calls to return the desired result, right? But how do I mock this?

I need the AuthenticationManager.authenticate to pretty much do nothing, as in case of a failed authentication it will throw an Exception, else it has the return type void. How do I tell my OrderService.save() to use my mocked AuthenticationManager.authenticate()?

And how do I set the AuthenticationManager up to either do nothing or throw an exception?

Can I tell Spring to inject a mocked AuthenticationManager that will do nothing / throw an Exception into my OrderService under test?

closed as not a real question by user8 Feb 14 '12 at 21:51

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center. If this question can be reworded to fit the rules in the help center, please edit the question.

  • Hi Pete, while it looks like you got what you were looking for from DXM's answer, in the future, please keep it to one specific issue per question: questions that are extremely broad and all-encompassing are not a fit for the Stack Exchange style of Q&A. – user8 Feb 14 '12 at 21:52
  • I see. Will try to stick to one question and get to the point in the future. I think I just had too many questions on the subject in general and didn't know where to ask them... – Pete Feb 15 '12 at 8:38

That is a really long question and I apologize but I only skimmed the additional thought parts.

I can recommend two excellent books:

They don't use the exact frameworks you mentioned but the advice they give you is very valuable and easily applies to any scenario with OOD. Both go into great detail of how to setup unit tests to isolate just the component being tested and how to set up mock/fake/stub objects to remote dependencies.

UPDATE (Response to below comment):

This answer is in response to OP's:

  • "I would like to have more of the described collaboration tests and contract tests" - contract tests are unit tests because OP wants to remove integration tests, and instead, test individual components against a well defined interface for that component. That is unit testing and precisely what those books are about.
  • "...where I can ask more detailed questions about how to go about correctly unit testing and maybe even refactoring existing code"
  • "Is there a general rule, when to use a mock and when a stub to isolate a class under test from it's collaborators?" - Both those books deal very heavily with setting up mock objects and using them properly
  • "Is there a book or a tutorial or example out there that takes a real world example and applies these ideas of isolated micro tests?" - Yes, this is called Unit Testing and the two books I listed are great at explaining this.

As OP pointed out, integration tests cause have many issues because they tend to become very complex and hard to write very fast, good alternative is unit test and testing each class in isolation against their public interfaces. In addition to unit tests, both of those books discuss integration testing as well and mention where it should and should not be used.

Furthermore, after skimming OP's question some more, it seems a lot of uncertainty revolves specifically around the use of mock objects and both of those books really do focus on how to properly use mock objects (although they don't mention Spring or specific mock frameworks)

  • 1
    He is not asking about unit tests, but about integration tests. Therefore I don't think you answered the question – BЈовић Feb 14 '12 at 15:32
  • @VJovic: my response is in the answer update. – DXM Feb 14 '12 at 17:48
  • Thanks for the detailed answers. Ordered the xUnit Book earlier. Hope it's as good as you make it sound ;) @VJovic: Does the question really look like I'm asking about integration tests on first sight? Because really I am asking about unit tests and am just referencing the conference video called "Integration tests are a scam" meaning: integration tests are (most of the time) not only not useful and problem solving but also dangerous. Please let me know, where exactly you found the post misleading. – Pete Feb 14 '12 at 19:22
  • btw I didn't downvote (on the contrary, I +1). @Pete Removing the “integration test scam” - Understanding collaboration and contract tests - this is your title. I might have misunderstood because you talked about integration tests, and then asked some unit tests questions. – BЈовић Feb 14 '12 at 20:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.