I've written a class that manages recipients on a MailChimp list, called MailChimpRecipient. It uses the MCAPI class, which is an third-party API wrapper.

http://apidocs.mailchimp.com/api/1.3/ http://apidocs.mailchimp.com/api/downloads/

I pass the MCAPI object into the constructor of the MailChimpRecipient object, so I have written unit tests using PHPUnit that test all of the logic in my own class (I am not testing the MCAPI class). I have 100% code coverage and all tests pass. This is done by mocking and stubbing the MCAPI object.

My next step was to write an integration test, also using PHPUnit, where I would construct the MailChimpRecipient fixture using a real MCAPI object, set up to use a real MailChimp list.

I have written what I think is an intergration test, which basically runs tests agains the public interface of the object, like:

public function testAddedRecipientCanBeFound()
    $emailAddress = 'fred@fredsdomain.com';
    $forename = 'Fred';
    $surname = 'Smith';

    // First, delete the email address if it is already on the list
    $oldRecipient = $this->createRecipient();

    // Add the recipient using the test data
    $newRecipient = $this->createRecipient();

    // Assert that the recipient can be found using the same email address

The "integration" test doesn't test any of the internals of the class - it just makes sure that given a real MCAPI object, it behaves as advertised.

Is this correct? Is this the best way to run an intergation test? After all, the internals have been tested with a unit test. Am I right in thinking that the integration test is there to test that it really works, according the way its behaviour is advertised?

To take it a step further, the MailChimpRecipient class implements an interface, which will also be implemented by other classes. The idea is to use a factory to pass different types of mailing list recipient objects to my code, which all do the same thing, albeit using different mailing list providers. Since my integration tests test the that interface, how about using it for all of the classes that implement the interface? Then, in the future, if I design a new class to be used interchangably, I can run the same integration test before inserting it into a project.

Does this sound reasonable? The unit tests test the internals of an object, intergration tests make sure that it behaves as advertised?

  • 4
    I think you have too much logic in your test. You run a lot of code until you do the assert. You probably want to test the deletion of an recipient first. But that's not answering your question, just a comment. – hakre Jan 4 '12 at 11:18
  • 1
    Well you should make use of the setUp function to establish the grounds to run your tests. If the input is undefined, well you can't really test. The input needs to be precise, strict and always the same. If a precondition of a test is not met, skip the test instead. Then analyze why it skips and if you need to add additional tests and/or the setUp is not done right. – hakre Jan 4 '12 at 11:33
  • 1
    Also don't hardcode test values inside a test of it's own probably but make those class members so they can be shared across tests (and changed at a central place) or use DataProvider (that's a function offering input as parameters to a test). – hakre Jan 4 '12 at 11:35
  • 1
    Input in the meaning of everything your test function operates on. As you test adding a recipient and you want to ensure that it does not exist already, you should at least assert the deletion in case it kicks in. Otherwise the precondition of your test is not ensured to be testable. – hakre Jan 4 '12 at 11:55
  • 1
    +1 for good question, but also voted to migrate to Programmers. Seems that's where questions on testing strategies belong – GordonM Jan 4 '12 at 13:10

When testing your code you should pay attention to three areas:

  • Scenario testing
  • Functional testing
  • Unit testing

Normally the amount of test you have in each category would have the shape of a pyramid, meaning a lot unit tests at the bottom, some functional tests in the middle, and just a few scenario tests.

With a unit test you mock out everything that the class under test uses and you test it in pure isolation (this is why it's important to make sure that inside your class you retrieve all dependencies trough injection so they can be replaced under test).

With unit testing you test all possibilities, so not only the 'happy path' but also all error conditions.

If you are completely certain that your all units work in isolation you write a couple of tests (functional tests) to make sure the units also work when combined. Then you write a scenario test, which tests the wiring between all the functional modules.

For example, suppose you're testing a car.

You could assemble the whole car and as a driver check every possible condition but that would be really hard to do.

Instead you would test a small part of the engine with all possibilities (unit test)

Then you test the whole engine (separate from the car) which would be a functional test.

As a last test, you put your key in, start the car and drive it to the parking lot. If that's working then you know that all parts (battery, fuel, engine, ..) are connected and since you tested them in isolation you can be pretty sure that the whole car is functioning correctly.

So in your case, you tested all error conditions and the happy path in your unit test and know you only have to an end-to-end test with the 'real components' to check if the wiring is correct.

A few other points,

  • Avoid conditional logic in your unit test. If you have to clean up, your using some kind of global state and the tests can suddenly influence each other.
  • Don't specify any data that's not relevant for the test. If I would change forename, or surname, would the test fail? Probably not because it's the email address that's important but because you mention it explicitly in your test, I can't be sure. Try looking at the Builder Pattern for building your test data and making it explicit what's really important.
| improve this answer | |
  • Thanks, that confirms alot of what I thought. Just to clarify - this is NOT a unit test. I've already written a unit test, which tests the object in complete isolation and has 100% code coverage of the object. This was meant to be an integration test, to make sure it works when I inject a real MCAPI object into it. I just need to delete any recipients that are added to the list - that's all the cleanup is, and it's implemented to ensure that none of the tests influence eachother. What would you suggest instead? – Lewis Bassett Jan 4 '12 at 12:34
  • 1
    Yeah! I understood you already did the unit tests. Does the MCAPI object keep track of the recipients and is that the cleanup you have to do? If it's the third parties 'problem', there is nothing you can do about in an integration test. If you on the other keep track of the list, you should make sure you're avoiding global data (and singletons) to make sure tests don't influence each other. In a perfect world, cleaning things up when a test starts/finishes, points to a design flaw but in the real world, you can't always avoid it. – Wouter de Kort Jan 4 '12 at 12:45
  • 1
    I would add that scenario testing is probably not really something PHPUnit is suited to. Yu might want to look at some tool you can run in a browser such as Selenium, or a tool that can simulate a browser, like jMeter. – GordonM Jan 4 '12 at 13:19
  • Thanks guys! There sure is a lot to learn when it comes to writing good testable code, isn't there. I've ordered myself a copy of this book: amazon.co.uk/…. Hopefully, what you've all said will make a bit more sense after I've read that. @Wouter, I'm just deleting a recipient because the test would have caused an email address to be added to the list. I'm deleting it so that the list is unaffected by that test. – Lewis Bassett Jan 4 '12 at 16:43
  • 1
    @LewisBassett I'm no Php developer but the xUnit Test patterns(amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054) is definitely a good read. Also the articles at misko.hevery.com/code-reviewers-guide are really interesting. – Wouter de Kort Jan 4 '12 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.