Should I use my own application's abstractions for test setup?

Question

When doing test setup, is it better to use low-level APIs to get my system in the right state, or use the same abstractions that my application would use? I'm specifically interested in what promotes long-term maintainability in large codebases.

Example (in Ruby, but my question is language-agnostic):

A User can buy an Item using the Sale class:

class User
  attr_accessor :join_source, :status

  def web_join!
    @join_source = 'web'
    @status = 'rookie'
  end

  def promote!
    @status = 'veteran'
  end

  def web_rookie?
    join_source == 'web' && status == 'rookie'
  end
end

class Sale
  attr_reader :item, :user

  def initialize(item, user)
    @item = item
    @user = user
  end

  def finalize
    item.owner == user
    if user.web_rookie?
      Email.send_web_rookie_receipt(user)
    end
  end
end

In testing Sale, I will need to create a user in various states. I can either user low-level APIs:

user = User.new
user.join_source = 'web'
user.status = 'rookie'
# test that the send_web_rookie_receipt goes out
user.status = 'veteran'
# test that the send_web_rookie_receipt does not goes out

Or the API that User exposes:

user = User.new
user.web_join!
# test that the send_web_rookie_receipt goes out
user.promote!
# test that the send_web_rookie_receipt does not goes out

To clarify, I am not asking about testing User in this scenario. I am asking about my tests for Sale, and whether the setup for testing Sale should use more abstract methods on User (such as `promote!) or more concrete, low level methods (attribute assignment).

Answer 1

From my point of view, this question is very much about trade-offs. Here are some points to think about when making your decision:

• If you use your high-level API and it breaks, all your tests break, too. When executing your tests, you'll see a lot of red tests and it might be more difficult to identify the root cause.
• Using a low-level API leads to more complex code in your test setup. This code is also subject to bugs. A big problem here is that you usually don't test your test setup so that these bugs are very hard to spot.
• Using the self-written high-level API makes your tests to integration-like tests, as you also use (and therefore test) the high-level API. Depending on the characteristics of your high-level API, this might slow down test significantly.

From my experience, I try to use the low-level API approach as often as possible to try to minimize the number of components under test. If you find your test setup to be very complex and brittle, this might also indicate a problem in your system's design (components are too tightly coupled). However, it's not an either-or-decision and I don't deny using the other approach, too, as it increases productivity (at least short-term) in cases where a test setup is just too cumbersome and can't be simplified easily.

Answer 2

IMHO for unit testing, normally one should use only parts of an API which a "client" of that class is expected to be used. So in this case, if the "low level API" as well as the "high level API" for the User class is available (a.k.a "public"), and both can be utilized for creating essentially the same test for Sale, use the one which allows it in an easier, simpler, less error-prone manner. Which one is "simpler" depends on the case, sometimes it can be easier to use the low level API to get an object into a certain state, sometimes it can be the high level API.

But what if using the high level API does lead to a more complex integration test, and using the low level API is more a real unit test (so the tests are not essentially the same)? For this case, consider to implement both, as discussed in this older SE.SE post.

If, however, only the "high level API" should be primarily used by the "client" code, and an attribute like join_source is only intended to be read, but not written, or it is expected to only hold a fixed set of values, you should probably question the fact if it is really a good design to have something like join_source fully public. Making this attribute "read only" for client code would make the API the less error-prone, and this would restrict also your tests automatically to the high level API.

I am not a Ruby expert, but I guess in a scripting language where everything is public by default, it is probably not uncommon to be sloppy and leave some attributes "public", even if they are not intended to be changed from outside. In a language like Java or C++ or C#, one would typically make only the "getters" for these attributes public, and leave the setters private right from the start. AFAIK you can do this in Ruby too.