33

According to rules of TDD unit tests are written before production code, but what about Integration tests that exercises interaction between concrete (non mocks) wired objects?

Should they be written before unit tests or after production code just to test the "wiring" ?

Note that I'm not talking about Acceptance or functional tests but Lower level integration tests.

7 Answers 7

53

The Rspec Book, among other BDD resources, suggests a cycle like this:

enter image description here

In essence, the process is:

While behaviour required
    Write an integration test for a specific behaviour
    While integration test failing
        Write a unit test to fulfil partial behavior
        While unit test failing
            Write code to make unit test pass
        Commit
        While refactoring can be done
            Refactor
            While unit test failing
                Write code to make unit test pass
            Commit
    Push

Disclaimer: There's no doubt in my mind that this leads to the best code and product, but it can be time-consuming. There are all sorts of difficulties around data and determinism, when it comes to saying that integration tests should always pass. It's not appropriate in all circumstances; sometimes you just have to get stuff out of the door.

That said, having an ideal process in mind is great. It gives you a point from which to compromise.

7
  • 2
    Thanks @pdr but I specified that I was not talking about Acceptance tests which are written before/at the beginning of an iteration, I'm interested in lower level integration tests.
    – Chedy2149
    Aug 16, 2013 at 10:11
  • @chedy2149: Akkh. Missed that comment. Before I remove my answer, I think you should be more specific about what you mean by "lower-level" in the context of integration tests.
    – pdr
    Aug 16, 2013 at 10:18
  • Lower-level : which behavior are not specified by users or clients and which are used to test classes/components interactions has expected by developers.
    – Chedy2149
    Aug 16, 2013 at 10:33
  • 6
    Actually, I don't think it makes a difference if you put "acceptance test" or "integration test" in that picture - it is an idealized view for any kind of tests on different abstraction levels. But IMHO the real problem in is not that this may be "time consuming" - the real problem is that writing integration tests "beforehand" against a public interface which is still beeing designed with the help of TDD is like shooting at a "moving target".
    – Doc Brown
    Aug 16, 2013 at 11:03
  • @DocBrown so your answer is after production code and before release?
    – Chedy2149
    Aug 16, 2013 at 11:26
11

Real project showed to me that it's not possible to write unit tests and then integration and even opposite direction is wrong :-) So, I usually write unit tests together with integration ones.

Why? Let me to write how I see both kinds of tests:

  1. Unit tests - In addition to Wikipedia and all know information, unit tests help you to narrow your design, improve your model, relations. The flow is simple: once you start to type new project/new component, most of time you are making some kind of PoC. When you are done, you always have long methods, long classes, non-coherent methods and classes, etc.

    Unit tests help you to remove these issues as when you do real unit testing using mocks (w/o dependency on other component) classes described above are un-testable. Basic sign of untestable code is large mocking part of tests because you are forced to mock many dependencies (or situations)

  2. Integration tests - correct and working tests say to you that your new component (or components) work together or with other components - this is usual definition. I've found that integration tests mostly help you to define flow how to use your component from consumer side.

    This is really important as it sometimes says to you that your API does not make sense from outside.

Well, what happen once I wrote unit tests and integration tests later?

I got nice classes, clear design, good constructor, short and coherent methods, IoC ready etc. Once I give my class/API to some consumer, e.g. developer from integration or GUI team, he was unable to use my API as it seems unlogic, weird. He was just confused. So I repaired API according to his point of view but it also required to rewrite many tests because I was pushed to change methods and sometimes even the flow how to use the API.

Well, what happen once I wrote integration tests and unit tests later?

I got exact flow, good usability. What I also have are large classes, non-coherent code, no logging, long methods. Spaghetti code

What's my advice?

I've learned following flow:

  1. Develop basic skeleton of your code
  2. Write integration tests that say whether it make sense from consumer point of view. Basic use-case is enough for now. The test obviously does not work.
  3. Write code along with unit tests for each class.
  4. Write the rest/missing of integration tests. It would be better to implement these test within #3 how you are improving your code.

Note that I've made small presentation about unit/integration testing, see slide #21 where the skeleton is described.

5

Unit Tests are used to test the smallest possible testable bit of software in an application and to test it's functionality. Each unit is tested separately before amalgamating them into parts or larger components of the application.

That's where Integration Tests come in:
They test these newly created parts that consist of the previously tested units during fitting these parts together. Best case would be to write the tests at this point while writing the application itself.

4
  • So your answer is after production code?
    – Chedy2149
    Aug 16, 2013 at 9:23
  • This doesn't answer the question. He's asking if production code is written after integration tests are written. Your answer can be taken either way.
    – Reactgular
    Aug 16, 2013 at 9:50
  • 1
    @MathewFoscarini - updated answer. Hope it becomes clearer now. Aug 16, 2013 at 10:13
  • With regard to unit tests, I would take issue with "smallest possible bit of software". Test what is in the contract (e.g. an object's public methods, a library's exported functions) because the contract defines what has to work. Other things are testable, yetdoing so is not only a waste of time but counterproductive.
    – itsbruce
    Aug 19, 2013 at 13:13
3

I tend to view integration tests as very similar to unit tests. In that I am treating a subset of the code as a black box. So integration tests are just a bigger box.

I prefer to write them before the production code. This has the advantage of helping me remember which pieces I haven't wired up yet or that I changed a detail in the objects interaction slightly.

1
  • There are different levels of testing: white box component testing, white box integration component testing. Component black box testing, integration black box testing. There is also integration system testing. Dec 11, 2016 at 23:42
2

Aside from acceptance tests, I tend to only write integration tests at the boundaries of an application, to verify that it integrates well with third-party systems or components.

The idea is to create adapter objects that translate from how the third party speaks to what your application needs, and test these translators against the real external system. Whether you do that test-first or test-last is I think less important than with your regular unit tests because

  • The design insight provided by TDD doesn't matter as much here since design is pretty much known in advance and there's typically nothing terribly complex involved, you just map things from one system to another.

  • Depending on the module/system you want to tackle, it can require a lot of exploration, configuration tinkering, sample data preparation, which takes time and doesn't fit really well in a short TDD feedback loop.

However, if you really feel more comfortable building your adapter incrementally in little secure steps, I'd definitely recommend going test-first, it can't hurt.

You can find examples of this approach here : http://davesquared.net/2011/04/dont-mock-types-you-dont-own.html (6th paragraph) http://blog.8thlight.com/eric-smith/2011/10/27/thats-not-yours.html

2
  • here you are talking about integration tests that verify interactions between "our system" and 3rd party libraries, how about testing interactions against a platform while developing a plug-in for example?
    – Chedy2149
    Aug 16, 2013 at 11:32
  • Though I have little experience with plugin development, I guess they might be different since they are by nature tightly coupled with the host application, so you may want to fully embrace that integration and decide you don't need an adapter layer. In that case I'd be really careful about test performance though - depending on the host application, calling its API directly in a large number of your tests can be very slow. If you're afraid of that, you can always resort to the "additional layer of abstraction" approach and use mocks + integration tests on adapters. Aug 16, 2013 at 11:51
2

So I was going to accept the first answer but it has been deleted.
To sum it up
In a given iteration :

  1. Write unit-test
  2. Write production code
  3. Write Integration tests to test interactions

Keep in mind integration testing while 1 and 2 to guarantee testability at the integration level.

Integration tests are not necessarily written end to end at step 3 they may be partially written between step 1 and 2.

2
  • 3
    This summing up completely ignores the iterative nature of the process. Once you have the API of your production code stable to some degree, you can start write integration tests.Then you can work on your production code again, and probably change or extend your integration tests. So in most cases you don't "write integration tests after production code", you typically do both to some degree in parallel. And actually, this depends also heavily on what kind of software you are working. "Black-and-white" thinking does not bring you further.
    – Doc Brown
    Aug 16, 2013 at 12:36
  • Good point.The answer seem to ignore the iterative nature of design through refactoring.
    – Chedy2149
    Aug 16, 2013 at 13:46
0

Unit tests test discrete blocks of code within your project.
Integration tests test how your code interfaces with other code: in other words, they test the interface of your code.

Write unit tests when developing code behind an interface.
Write integration tests when developing the interface or any code that implements the interface.

This means that you will sometimes write integration tests very late in a project, because the majority of the work is behind the interface: for example, a compiler, a particular webservice that implements several layers of logic or .. something that involves a lot of internal logic.

However, if you're implementing a set of REST services or refactoring the data model and adding support for XA transactions, then you're going to start developing integration tests almost immediately, because most of your work is centered around the interface, whether it's the REST API or how the program uses the data model.

5
  • Would you agree to say that unit tests are whitebox testing and integration tests are blackbox testing?
    – Chedy2149
    Aug 22, 2013 at 8:06
  • Unfortunately, that depends. Technologies for integration testing have made enormous improvements in recent years (at least in the Java world) so that I can test 1 class -- but on an application server. The question then becomes, where's the line between unit tests and integration tests? Is an integration test when you test your code interfacing with other technologies, or is an integration test when you test your whole application -- but not necessarily in the environment that it's meant to run in?
    – Marco
    Aug 22, 2013 at 8:43
  • In short, in some cases, sure integration tests are black box testing -- but not in all cases.
    – Marco
    Aug 22, 2013 at 8:44
  • FYI: Wiki defines integraion testing as "the phase in software testing in which individual software modules are combined and tested as a group"
    – Marco
    Aug 22, 2013 at 8:55
  • Exactly, Marco. There are integrations on each component level. Code level, application level, system level. Dec 11, 2016 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.