108

My friends and I have been struggling to classify exactly what is an integration test.

Now, on my way home, I just realised, that every time I try to give a real world example of an integration test, it turns out to be an acceptance test, ie. something that a business person would say out loud that specs out what the system should be delivering.

I checked the Ruby on Rails documentation for their classification of these testing types, and it's now completely thrown me.

Can you give me a short academic description of an integration test with a real world example?

  • 76
    BTW, when you have a noun phrase ("Me and some friends") you need to be careful which first-person singular you use. Here's the test. Drop the friends and see if it still works. "Me have been struggling" vs "I have been struggling". This test tells you that it's "I and my friends..." And -- to be polite -- we list other first. "My friends and I". Testing is important. – S.Lott Feb 15 '11 at 20:01
  • 57
    I think S.Lott just gave you a grammar->society integration test. – Jordan Feb 15 '11 at 21:13
77

At the moment I like this statement : "It’s not important what you call it, but what it does" made by Gojko Adzic in this article.

You really need to specify with the people talking about the tests what you intend to test.

There are a lot of people having different views, depending on what their role is.

For testers a general accepted test methodology in the Netherlands is TMap. TMap makes the following distinction.

  • unit test
  • unit integration test
  • system test
  • system integration test
  • acceptance test (all kinds/levels)
  • functional acceptance test
  • user acceptance test
  • production acceptance test

They have more specific kind of tests that can be performed within the above mentioned tests. Look at this word doc for an overview.

Wikipedia also has a nice overview.

The book the pragmatic programmer says:

  • a unit test is a test that exercises a module
  • integration tests show that the major parts of a system work well together

Looking at these different sources and putting in some of my own experiences and opinions I would start by making distinctions by three categories

  • who does the testing in general
  • what is tested
  • what is the goal of the test

    • Unit test: test logic in classes by programmers to show code level correctness. They should be fast and not dependend on other parts of the system that you don't intend to test
    • Functional acceptance test: test use case scenario's on a limited (specially created) data set done by the test department to show that every specified scenario works as specified.
    • User acceptance test: test use case scenario's on production like data done by representatives of the users to make them formally accept the application
    • Integration test: Test communication paths between different parts of the module done by the test department or by developers to show that all modules work correctly together.

My list above is just a start and a suggestion but I really do think: "It’s not important what you call it, but what it does"

Hope this helps.

26-10-2016 Edit: Just recently a very nice introduction was placed on YouTube Unit tests vs. Integration tests - MPJ's Musings - FunFunFunction #55

  • 3
    +1 For the "it doesn't matter what you call it". Sadly there is no universal definition of any kind of test. Even the good ol unit-test is a bit variable. Is testing the DOM for a web app considered a unit test? Some say yes, some say no. – Laurent Bourgault-Roy Jan 16 '14 at 20:43
  • 2
    The link to the word doc is not available. – Paul Rougieux Nov 2 '15 at 13:32
  • 6
    "It's not important what you call it" is applicable to all of computer science, and in fact almost all areas. I find a lot of the heated arguments people get into can be boiled down into "we're arguing over the definition of an arbitrary phrase". – gardenhead Oct 16 '16 at 23:07
  • +1 agree on the definitions : I've been in a place where "Unit Test" ment that the programmer tried the system with at least one sample input... manually... and visually checked the output if it "seemed right". No contract as to what was expected, no controlled inputs etc – Newtopian Oct 17 '16 at 14:49
  • The link to Gojko is returning a 404. You can access an archive here: web.archive.org/web/20150104002755/http://gojko.net/2011/01/12/… – Eduardo Copat Oct 24 '16 at 7:25
31

integration test, it turns out to be an acceptance test

Obviously.

These two are almost the same thing. But there are some slightly different dimensions to the test definition.

Integration == the system as a whole.

Acceptance == the system as a whole.

The only difference -- and this is subtle -- is the definition of the test cases.

Integration == test cases to test the depth and degree of integration. Does it work for all edge cases and corner cases? Test cases tend to be technical, written by designers and coders.

Acceptance == test cases to exercise just the end-user-focused 80% of the feature set. Not all the edge and corner cases. Test cases tend to be non-technical, written by end-users.

  • 7
    Only thing that I would add to this is that integration tests can also test just part of the system, but more than one piece at a time. Any time you are looking for bugs caused by two or more parts of the system working in unison (integrated together), you are integration testing. Integration runs from two components real and everything else mocked, to the entire application suite working together, and may even go so far as checking for integration with other applications (e.g., "How does MS Office work with Internet Explorer?"). – Ethel Evans Feb 15 '11 at 20:03
  • 1
    @Ethel Evans: Good point. The test will still be blurry between integration and acceptance, even if only a portion of the system is involved. The testing occurs at a high-enough level that acceptance and integration will feel similar. – S.Lott Feb 15 '11 at 20:08
  • 3
    Integration tests certainly do not (and probably should not) test "the system as a whole". Anywhere that two or more components are tested together, in particular when testing against external components (database, network etc.) you are doing integration testing. Because of the high costs of "whole system" testing you want to avoid this as much as possible, instead look to do more partial-system integration tests (see Test pyramid – Schneider Feb 27 '14 at 12:44
  • 1
    @Schneider says it well, integration tests should not test "the system as a whole". Such tests would be considered "end-to-end tests" or "system tests", depending on what scope you consider in your project. End-to-end tests might cover a dataflow "as a whole" that runs trough multiple systems, and system tests just "one system as a whole". – RoyB Oct 14 '16 at 15:02
  • Here's how I define them to avoid the confusion around the broad use of "Integration" test. Unit tests -> tests smallest unit of work, a method in a class, that does not call out to any other code outside of that method (mocking out dependencies if needed) Integration tests -> tests larger in scope from unit tests where they can and should test layers of an application working together, but NOT the whole application deployed somewhere.) Functional Tests/acceptance test -> tests that test a deployed version of app – Kevin M Jan 12 '17 at 16:04
15

I personally like to think of an integration test as of a feature test when every component of the system is real, no mock objects.

A real repository, a real database, a real UI. You test specific functionality when the system is fully assembled and is like it is supposed to be when deployed.

  • 4
    I agree except it doesn't need to be "fully assembled" system. You can (and should) do integration testing with subsets of the full system as this is usually cheaper/easier. – Schneider Feb 27 '14 at 12:48
  • Integration between 2 units/components can also be "integration tested" while the rest is still mocked. Hence, many integration testing frameworks allow mocking :) – RoyB Oct 14 '16 at 15:06
  • 1
    I have tests in a Spring Boot app that tests the front end (api contract) within a Spring container but mocks out the repository/data layer. So it's using mocked repositories/data, but merging the controller layer and doing the jackson marshalling, etc. Integration test. – Kevin M Jan 12 '17 at 16:06
7

In my (I admit) little experience, I understood that the word integration can really create misunderstandings: really, it's difficult to find something completely isolated in a system, some elements will need some integration for sure.

Thus, I got accustomed to make the following distinctions:

  • I use unit test to identify, document and stress all the behaviors the class I'm testing is responsible to accomplish.
  • I'm doing integration tests whenever I have a component (maybe more than one) in my system which is having some conversation with another "external" system. (continues below...)
  • I implement an acceptance test to define, document and stress a certain workflow which is expected by the system.

In the integration test definition, by external I meant system which are out of my development range: I cannot change immediately the way they behave, for any reason. It could be a library, a component of the system which cannot be changed (i.e. it is shared with other projects in the company), a dbms, etc. For these tests I need to set up something very similar to the real environment the system will work in: an external system must be initialized and set to a certain state; realistic data must be registered in the db; etc.

Instead, when I'm doing acceptance testing I fake things: I'm working on something different, I'm working on the specifications of the system, not on it's ability to collaborate with external entities.

This really is a narrower view compared to what KeesDijk described earlier, however I suppose the projects I worked on till now were small enough to let me this level of simplification.

5

An integration test verifies that components of a complex system (e.g. software, aircraft, power plant) are working together as designed.

Let's imagine we're talking about an aircraft (with software it's more abstract, and difficult to make the difference). The integration tests include, verifying:

  • correct interaction between some components. Example: when pressing on the start button, the engine starts and propeller reaches the expected rotation speed (aircraft still stays on ground)
  • correct interaction with external components. Example: check that the embedded radio can communicate with a stationary radio (aircraft still on ground)
  • correct interaction between all involved components, so that system as a whole works as expected. Example: a crew of test pilots and engineers starts the plane, and fly with it (they all wear parachutes...).

The integration test addresses a technical issue, namely that the system works despite its subdivision into components. In software the components can be use cases, modules, functions, interfaces, libraries, etc...

The acceptance test verifies that the product is fit for purpose. They are in principle performed by the customer. Taking the aircraft analogy, they include verifying that:

  • envisaged business scenarios lead to the expected result in an almost real situation. Example: rehearse a boarding with test passengers to check that staff can monitor the boarding as expected with the operating procedures. Some scenarios could be so simple that they would look like unit test, but they are performed by the user (e.g. try the electrical plugs with the companies' equipment).
  • the system works in an almost real business situation. Example: make an empty test flight between two real destinations, with newly trained pilots from the airline to check that fuel consumption is as promised.

The acceptance test addresses more a responsibility issue. In a client/supplier relationship it can be a contractual responsibility (compliance with all requirements). But in any case it's also the responsibility of the using organization to ensure that their duties can be carried on with the system and to prudently prevent any unforeseen issue (e.g. like this railways corporation that discovered during acceptance tests that they had to shorten some quais because the new wagons were 5 cm too large - no joke !).

Conclusions: Integration and acceptance tests are overlapping. They both intend to show that the system as a whole works. However the "whole" could be larger for the customer (because the system may itself be a part of a larger organizational system), and more technical for the system integrator:

enter image description here

1

Integration testing is nothing but checking the connection and correctness of data flow between two of more modules.

For Example: When we compose a mail(one module) and send it to some valid user ID(second module), the integration testing is to check whether the sent mail is there in the sent items.

  • 3
    Welcome to Programmers. What does your answer add that hasn't already been provided by the existing answers? Programmers.SE isn't like traditional forums. It focuses on high quality Q&A instead of lots of chatter. Please have a look at the tour page for more information on how the site operates. – GlenH7 Jan 16 '14 at 21:49
0

One practical definition of an integration test is: Any test that requires interaction with something out-of-process.

For example:

  • The file system
  • The network
  • A database
  • An external API

There exists a contract of sorts between your process and the external world, and minimally verifying that contract should be the goal of an integration test. i.e. it should do no more than verify the contract. If it does then you are moving toward the system/end-to-end space.

Unit tests are able to test all logic within your process boundary, and they can do it easily precisely because of the lack of dependencies on the slow/fragile/complex "external world".

Whilst there are integration tests this definition doesn't cover (hence why I called it a practical definition) I think they are much less common/useful.

N.B. Strictly speaking, yes this definition would cover system/end-to-end tests too. In my philosophy they are a form of 'extreme' integration test, hence why their names emphasise another aspect. In the other direction, a unit test can be considered an integration test of zero components i.e. All tests can be considered to lie somewhere on the integration spectrum, integrating between 0-n components :-)

  • When you have two units, you test each with a unit test. When those two units integrate with each other, you test the integration with an integration test. They don't have to be "out of process", and these kinds of tests are extremely common. – Bryan Oakley Oct 24 '16 at 13:07
  • You're absolutely correct - things don't have to be out of process to be an integration test. That's why I tried to be clear that my answer was more a "rule of thumb" (but perhaps failed). I disagree that integration tests between units are "extremely common" though. Integration tests with out of process are much more common in my experience and are frequently highly valuable, hence why my answer emphasises that aspect of integration testing. – Schneider Oct 25 '16 at 0:02

protected by gnat Oct 17 '16 at 6:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.