32

Most unit testing tutorials/examples out there usually involve defining the data to be tested for each individual test. I guess this is part of the "everything should be tested in isolation" theory.

However I've found that when dealing with multitier applications with a lot of DI, the code required for setting up each test gets very long winded. Instead I've built a number of testbase classes which I can now inherit which has a lot of test scaffolding pre-built.

As part of this, I'm also building fake datasets which represent the database of a running application, albeit with usually only one or two rows in each "table".

Is it an accepted practice to predefine, if not all, then the majority of the test data across all the unit tests?

Update

From the comments below it does feel like I'm doing more integration than unit testing.

My current project is ASP.NET MVC, using Unit of Work over Entity Framework Code First, and Moq for testing. I've mocked the UoW, and the repositories, but I'm using the real business logic classes, and testing the controller actions. The tests will often check that the UoW has been committed, e.g:

[TestClass]
public class SetupControllerTests : SetupControllerTestBase {
  [TestMethod]
  public void UserInvite_ExistingUser_DoesntInsertNewUser() {
    // Arrange
    var model = new Mandy.App.Models.Setup.UserInvite() {
      Email = userData.First().Email
    };

    // Act
    setupController.UserInvite(model);

    // Assert
    mockUserSet.Verify(m => m.Add(It.IsAny<UserProfile>()), Times.Never);
    mockUnitOfWork.Verify(m => m.Commit(), Times.Once);
  }
}

SetupControllerTestBase is building the mock UoW, and instantiating the userLogic.

A lot of the tests require having an existing user or product in the database, so I've pre-populated what the mock UoW returns, in this example userData, which is just an IList<User> with a single user record.

  • 4
    The problem with tutorials/examples is that they need to be simple, but you can't show solution to a complex problem on a simple example. They should be accompanied by "case studies" describing how the tool is used in real projects of reasonable size, but they rarely are. – Jan Hudec Sep 27 '13 at 6:18
  • Maybe you could add some small examples of code you have which you are not totally happy about. – Luc Franken Sep 27 '13 at 7:58
  • If you need a lot of setup code to run a test you risk running a functional test. If the test fails when you change code but there is nothing wrong with the code. It's definitely a functional test. – Reactgular Sep 27 '13 at 11:06
  • The book "xUnit Test Patterns" makes a strong case for reusable fixtures and helpers. Test code should be just as maintainable as any other code. – Chuck Krutsinger Oct 2 '13 at 18:00
  • This article may be helpful: yegor256.com/2015/05/25/unit-test-scaffolding.html – yegor256 May 25 '15 at 17:06
25

Ultimately, you want to write as little code as possible to get as much outcome as possible. Having a lot of the same code in multiple tests a) tends to result in copy-paste coding and b) means that if a method signature changes you can end up having to fix a lot of broken tests.

I use the approach of having standard TestHelper classes that provide me with a lot of the data types that I routinely use, so I can create sets of standard entity or DTO classes for my tests to query and know exactly what I will get each time. So I can call TestHelper.GetFooRange( 0, 100 ) to get a range of 100 Foo objects with all their dependent classes/fields set.

Particularly where there are complex relationships configured in an ORM type system which need to be present for things to run correctly, but aren't necessarily significant for this test that can save a lot of time.

In situations where I'm testing close to the data level, I sometimes create a test version of my repository class that can be queried in a similar way (again this is in an ORM type environment, and it wouldn't be relevant against a real database), because mocking out the exact responses to queries is a lot of work and often only provides minor benefits.

There are some things to be careful of, though in unit tests:

  • Make sure your mocks are mocks. The classes that perform operations around the class being tested must be mock objects if you are doing unit testing. Your DTO/entity type classes can be the real thing, but if classes are performing operations you need to be mocking them- otherwise when the supporting code changes and your tests start failing, you have to search for a lot longer to figure out which change actually caused the problem.
  • Make sure you are testing your classes. Sometimes if one looks through a suite of unit tests it becomes apparent that half of the tests are actually testing the mocking framework more than the actual code that they are supposed to be testing.
  • Don't reuse mock/supporting objects This is a biggie- when one starts trying to get clever with code supporting unit tests it is really easy to inadvertently create objects that persist between tests, which can have unpredictable effects. For example, yesterday I had a test that passed when run on its own, passed when all tests in the class were run, but failed when the entire test suite was run. It turned out there was a sneaky static object way off in a test helper that, when I created it, would definitely never have caused a problem. Just remember: At the start of the test, everything is created, at the end of the test everything is destroyed.
10

Whatever makes the intent of your test more readable.

As a general rule of thumb:

If the data is part of the test (eg. Should not print rows with a state of 7) then code it in the test, so that it's clear what the author intended to happen.

If the data is just filler to make sure it has something to work with (eg. Should not mark record as complete if processing service throws exception) then by all means have a BuildDummyData method or a test class that keeps irrelevant data out of the test.

But note that I'm struggling to think of a good example of the latter. If you have many of these in a unit-test fixture, you probably have a different problem to solve ... perhaps the method under test is too complex.

  • +1 I agree. This smells like what he's testing is to tightly coupled for unit testing. – Reactgular Sep 27 '13 at 12:16
5

Different methods of testing

First define what you are doing: Unit testing or integration testing. The number of layers is irrelevant for unit testing since you only test one class most likely. The rest you mock out. For integration testing it is inevitable that you test multiple layers. If you have good unit tests in place the trick is to make the integration tests not too complex.

If your unit tests are good you don't have to repeat testing all details when doing integration testing.

Terms we use, those are a bit platform dependent, but you can find them in almost all test / development platforms:

Example application

Depending on the technology you use the names may differ, but I will use this as an example:

If you have a simple CRUD application with model Product, ProductsController, and an index view which generates an HTML table with products:

The end result of the application is showing an HTML table with a list of all products which are active.

Unit testing

Model

The model you can test quite easily. There are different methods for it; we use fixtures. I think that's what you call "fake datasets". So before each test is run we create the table, and put in the original data. Most platforms have methods for this. For example, in your test class, a method setUp() which is run before each test.

Then we run our test, for example: testGetAllActive products.

So we test directly to a test database. We do not mock out the datasource; we make it always the same. This allows us for example to test with a new version of the database, and any query issues will come up.

In the real world you cannot always follow 100% single responsibility. If you want to do this even better you could use a datasource which you mock. For us (we use an ORM) that feels like testing already existing technology. Also the tests become much more complex, and they don't really test the queries. So we keep it this way.

The hard coded data is separately stored in the fixtures. So the fixture is like an SQL file with a create table statement and inserts for the records we use. We keep them small unless there is a real need to test with lots of records.

class ProductModel {
  public function getAllActive() {
    return $this->find('all', array('conditions' => array('active' => 1)));
  }
}

Controller

The controller needs more work, because we don't want to test the model with it. So what we do is mock the model. That means: We test: index() method which should return a list of records.

So we mock the model method getAllActive() out and add fixed data in it (two records for example). Now we test the data the controller sends to the view, and we compare if we really get those two records back.

function testProductIndexLoggedIn() {
  $this->setLoggedIn();
  $this->ProductsController->mock('ProductModel', 'index', function(return array(your records) ));
  $result=$this->ProductsController->index();
  $this->assertEquals(2, count($result['products']));
}

That's enough. We try to add as little functionality to the controller because that makes testing hard. But of course there is always some code in it. For example, we test requirements like: Show those two records only if you are logged in.

So, the controller needs one mock normally and a small piece of hardcoded data. For a login system maybe another one. In our test we have a helper method for it: setLoggedIn(). That makes it simple to test with login or without login.

class ProductsController {
  public function index() {
    if($this->loggedIn()) {
      $this->set('products', $this->ProductModel->getAllActive());
    }
  }
}

Views

Views testing is hard. First we separate out logic which repeats. We put it in Helpers and test those classes strict. We expect always the same output. For example, generateHtmlTableFromArray().

Then we have some project specific views. We don't test those. It is not really desired to unit test those. We keep them for integration tests. Because we took out a lot of the code into views we have a lower risk here.

If you start testing those you likely need to change your tests every time you change a piece of HTML which is not useful for most projects.

echo $this->tableHelper->generateHtmlTableFromArray($products);

Integration testing

Depending on you platform here you can work with users stories, etc. It could be webbased like Selenium or other comparable solutions.

Generally we just load the database with the fixtures and assert which data should be available. For full integration testing we generally use very global requirements. So: Set product to active and then check if the product becomes available.

We don't test everything again, like whether the right fields are available. We test the bigger requirements here. Since we do not want to duplicate our tests from the controller or view. If something is really key / core part of you application or for security reasons (check password is NOT available) then we add them to ensure it's right.

The hard coded data is stored in the fixtures.

function testIntegrationProductIndexLoggedIn() {
  $this->setLoggedIn();
  $result=$this->request('products/index');

  $expected='<table';
  $this->assertContains($expected, $result);

  // Some content from the fixture record
  $expected='<td>Product 1 name</td>';
  $this->assertContains($expected, $result);
}
  • This is a great answer, to an entirely different question. – pdr Sep 27 '13 at 8:08
  • Thanks for the feedback. You might be right that I did not mention it too specific. The reason of the verbose answer is because I see one of the most difficult things when testing in the question asked. The overview of how testing in isolation fits with the different kinds of tests. That's why I added in every part how the data is handled (or separated out). Will take a look to see if I can get it more clear. – Luc Franken Sep 27 '13 at 8:24
  • The answer has been updated with some code examples to explain how to test without calling all sorts of other classes. – Luc Franken Sep 27 '13 at 9:39
4

If you are writing tests that involve a lot of DI and wiring, up to using "real" datasources, you probably left the area of plain unit testing and entered the domain of integration testing.

For integration tests, I think, it is not bad idea to have common data setup logic. The main goal of such tests is to prove that everything is correctly configured. This is rather independent of the concrete data sent through your system.

For Unit tests on the other hand, I would recommend to keep the target of a test class a single "real" class and mock everything else away. Then you should really hard-code the test data to make sure you covered as many special/previous-bug paths as possible.

To add a semi-hard-coded/random element to tests, I like to introduce random model factories. In a test using an instance of my model, I then use these factories to create a valid, but completely random model object and then hard-code only the properties that are of interest for the test at hand. This way you specify all relevant data directly in your test, while saving you the need of also specifying all irrelevant data and (to a certain degree) test that there are no unintended dependencies on other model fields.

-1

I think it is pretty common to hard code most of the data for your tests.

Consider a simple situation where a particular data set causes a bug to occur. You might specifically create a unit test for that data to exercise the fix and ensure that the bug doesn't come back. Over time your tests will have a set of data that covers a number of test cases.

Predefined test data also allows you to build a set of data that covers a wide and known range of situations.

That said, I think there is also value in having some random data in your tests.

  • Did you really read the question and not just the title? – Jakob Sep 27 '13 at 7:14
  • value in having some random data in your tests -- Yeah, cause there's nothing quite like trying to figure out what happened in a test the one time it fails each week. – pdr Sep 27 '13 at 7:42
  • There is value in having random data in your tests for hazing/fuzzing/input tests. But not in your unit tests, that would be a nightmare. – glenatron Sep 27 '13 at 10:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.