2

When using factories, although more work is required up front, you know that you will be setting the correct pre-conditions for your test (up to what you have written.) However, when you use fixtures, although no work is required to setup the database state, someone (including the developer of the test) could easily change the fixtures by accident in some way in the future and render the test useless (e.g. make it always pass no matter what!)

How should a develop guard against this to protect the integrity of the test in the future? Should the pre-conditions be asserted before the test runs? Is this considered best practice?

8
  • 1
    I don't understand. Yes, someone can change a fixture. But someone can also change the behavior of a factory such that tests are invalid. A good example is changing the default state if none is provided - tests could be written with one state but end up with another.
    – Thomas Owens
    Apr 28, 2018 at 0:36
  • Fair enough, but it seems a bit less likely in my opinion. Either way, is checking preconditions as a part of the test considered the best way to guard against this? Apr 28, 2018 at 0:52
  • I don't fully understand what problem you are trying to solve. It feels like adding more code to assert preconditions will only make the tests harder to read and that well structured tests and good change management is all you need. Did this situation happen to you? Or are you trying to preemptively solve a problem that may or may not exist?
    – Thomas Owens
    Apr 28, 2018 at 0:59
  • @ThomasOwens the problem of testing some condition that may or may not be true in the future. For example say you are testing a user doesn't have any associated Image instances in a specific scenario, but then someone changes the fixtures so the user doesn't have any of the associations in the first place. Then the test would always pass. I'm wondering how people guard against this, because it must be relatively common? I can't think of any way besides either placing comments, making some documentation, or asserting it's true before running the test. Apr 28, 2018 at 1:12
  • If I understand correctly, you mean the fixtures make the user 'lose' the image association implicitly and that's what you're asserting on ? Then it looks like to fixture ought to be tested as it contains logic. Then someone breaking the fixture's behavior would break the tests too. Apr 28, 2018 at 1:30

1 Answer 1

2

If a specific property of the environment is important for that test, then yes, do verify that precondition directly in the test. In particular, this is helpful when the system under tests performs a kind of state transition.

For example, I am writing a user account management system. I want to test that I can add new users. In order to test whether a new user has been created, it makes sense to ensure the user didn't exist previously. E.g. expressed in Cucumber notation:

Scenario: adding a new user
  Given I have an account management system
    And no user named "Fred" exists
   When I add a user named "Fred"
   Then a user named "Fred" exists

or more xUnit-ly:

void test_addingANewUser() {
  assertNull(accounts.getUserByName("Fred"));

  accounts.createUser("Fred");

  assertNotNull(accounts.getUserByName("Fred"));
}

I think verifying these preconditions is a perfectly clear thing to do, regardless of how you create the system state prior to the test. Without this precondition, the test case is incomplete and depends on some setup code for the full details. With this precondition, the test is more self-contained and thus more maintainable.

5
  • Thanks, do people mainly do this with any sort of shared state not set up by tests? If not, do people just assume the fixtures will never change in a way that affects their tests negatively, etc.? Apr 28, 2018 at 17:02
  • 1
    I think people would usually assume that their fixture doesn't start breaking their tests. But unless these aspects of the fixture are tested, that's a bad assumption. It is normal for use cases to state preconditions. Therefore, it is sensible to verify preconditions in test cases that are derived from such use cases.
    – amon
    Apr 28, 2018 at 18:12
  • (Personally, I prefer setting up a completely new state within each test case, although that's not always feasible (especially for end to end tests). But even if I've just explicitly created some state, I will test relevant preconditions to make the test more explicit.)
    – amon
    Apr 28, 2018 at 18:13
  • My thoughts exactly, especially in big projects with many developers, but I am still a new developer and don't know the best practices. Hopefully more devs can share their experiences. Apr 28, 2018 at 20:12
  • The following examples supplement @amon answer: github.com/internetee/registry/blob/… Apr 23, 2019 at 11:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.