Test Doubles, Mocks and Stubs - when not to use & why, for Ruby, Rspec

Question

I've learned about mocking and stubbing and I've seen how they can help me create great test suites that run blindingly fast and thus speed up my development process hugely.

However I've also seen the downside when there is an issue that is database centred and wasn't tested for (test doubles or not).

So should I have a subset of tests that do test such connectivity at least and some simple operations, for example creating a record in the primary transaction detail table, even if most of the foreign keys are stubbed out?

What experience have people found with this issue and what balance has worked in various situations?

I normally post on Stack Overflow but this seemed much more conceptual. It might still be a bit subjective so maybe the question can be improved.

The closest similar question I could find was When should I use mock objects? but it didn't feel like it was really the same question and the answers felt further off. Perhaps my question is more about when NOT to mock. ...yeah, so I've updated my title.

I also found this helpful: TDD: Mocking out tightly coupled objects

Answer 1

I see unit tests as routines that test pieces of code that are in a code base that you manage. Mocking and stubbing helps focus only on the flow and logic of your code, which is what you are testing in your unit tests.

Whether or not an outside service like a database (which is not inherently part of your code) actually works is not important for unit tests, but rather for integration tests.

So should I have a subset of tests that do test such connectivity at least and some simple operations, for example creating a record in the primary transaction detail table, even if most of the foreign keys are stubbed out?

Only if you have implemented routines to do simple database operations, should you tests whether those routines work like you expect them to, in unit tests. But then you would still stub out the database because you are not particularly interested in whether the database actually works, you are testing whether the logic in your code works.

Answer 2

The first and most important test of a component is to test the external interfaces. Write those tests first, and make sure they are comprehensive. Then add some more. Then review them, find out which cases are missing, and add them.

Then add some more.

Write lower level (i.e. class-as-unit) tests only when:

1. your higher-level (i.e. component-as-unit) tests are failing, and you can't figure out why.
2. you have some budget left after you have written every possible external interface test.
3. you have a resuable, generic or just plain over-engineered utility class or function.
4. it is explicitly required by the customer.

For testing external interfaces, every interface is going to be either an input, an output, or a collaborator (like a database).

For a component-as-unit test:

• test the input interfaces; call the methods, assert the return values.
• mock the output interfaces; verify the expected methods are called for a given test case.
• fake the collaborators; provide a simple but working implementation

If you do that properly and cleanly, you barely need a mocking tool; it only gets used a few times per system. And unit tests should run in milliseconds (at least unless your code is horrendously slow, in which case perhaps you should fix it). Integration test then replace the fakes and mocks, one by one when you first do it, or all except user interfaces for a regression test.

Answer 3

So should I have a subset of tests that do test such connectivity at least and some simple operations, for example creating a record in the primary transaction detail table, even if most of the foreign keys are stubbed out?

Unit tests can and should never replace integration tests. That is, when testing the data layer you should always use integration tests as there is no fake/stub in the world that can act as the database server do.

So the answer is yes, you should always have tests that make sure that your code access the database in a correct way.

Answer 4

It boils down to the simple fact that you have to decide which cases you need to cover.
If you need to be sure of the database connection, you have to test it.

Tests are separated in groups based on what they test (unit, integration and system), where only system tests would make real calls to a database (or other external services).
Each type of test covers specific parts of the system, and ignores (or stubs out) the other parts.

I have put a simplified schematic illustration below. components inside the green box are tested, whereas components outside the green box are stubbed (faked, mocked, etc.)

for ruby/rspec there are tools (e.g. capybara) to assist with system tests, frequently called acceptance tests, when they are used to demonstrate a system does what it is supposed to.