1

My understanding of mocking vs not mocking is that mocking too much creates brittle tests that need to be changed all the time, while on the other hand less mocking better verifies that the system works as it's supposed to all the way through.

I can see the value of not mocking too much in an untyped language since, but there are some parts that I have trouble understanding when it comes to a layered architecture in a typed language. I hope it's possible to understand what I'm asking here, but if not don't hesitate to comment!

Say I have the following controller and accompanying service classes in an MVC application. (This example is in C# but I hope it's possible to follow even without knowledge.)

public class ApplicationController
{
    private IApplicationService _service;
    public ApplicationController(IApplicationService service)
    {
        _service = service;
    }
    [HttpPost]
    public ActionResult LoginUser(string id)
    {
        var result = _service.LoginUser(id);
        if (!result) return Unauthorized();
        return Ok();
    }
}

public class ApplicationService
{
    private DbContext _dbContext;
    private HttpContextAccessor _httpContextAccessor;
    public void LoginUSer(DbContext dbContext, HttpContextAccessor httpContextAccessor)
    {
        _dbContext = dbContext;
        _httpContextAccessor = httpContextAccessor;
    }
    public bool LoginUser(string id)
    {
        var user = _dbContext.Users.Get(id);
        if (user == null) return false;
        if (!user.Active) return false;
        _httpContextAccessor.HttpContext.User.Identity.Id = user.Id;
        return true;
    }
}

When testing the ApplicationController I can think of two simple tests: - If result is false, return Unauthorized - If result is true, return Ok

Not mocking anything - tight coupling?

Since my approach in this example is to not use any mocks, I need to somehow simulate ApplicationService returning false. One way is to seed the database with a user that has the property Active = false.

[Fact]
public void LoginUser_UserNotActive_ReturnsUnauthorized()
{
    var sut = new ApplicationController();
    _dbContext.Add(new User { Id = "123", Active = false });
    var result = sut.LoginUser("123");
    Assert.IsType<UnauthorizedResult>(result);
}

This is where it gets tricky for my understanding. In order to create this test, my ApplicationController test has to "know" in which instances the ApplicationService will return false and set up the environment properly, which creates coupling, right? Let me illustrate what I mean.

Now consider me writing tests for the ApplicationService. A simple test would be the following.

public void LoginUser_UserNotActive_ReturnsFalse()
{
    var sut = new ApplicationService();
    _dbContext.Add(new User { Id = "123", Active = false });
    var result = sut.LoginUser("123");
    Assert.False(result);
}

With this coupling which (from my understanding) stems from not writing black-box tests, if a later requirement says that you can login users which are active, I have to change both tests! Furthermore if I write an ApplicationController test to test UserNotActive, I have to test for UserDoesNotExist too, right? But at that point I'm duplicating tests, just in different layers.

Wouldn't it be better with mocking..?

If on the other hand I wrote the controller test using mocks, it would maybe look like this.

public void LoginUser_UserNotActive_ReturnsUnauthorized()
{
    var serviceMock = Substitute.For<ApplicationService>();
    serviceMock.LoginUser(Arg.Any<string>()).Returns(false);
    var sut = new ApplicationController(serviceMock);
    var result = sut.LoginUser("someId");
    Assert.IsType<UnauthorizedResult>(result);
}

With this test, if I change the requirements to be able to login with an inactive user, I would not need to change the ApplicationController test since I'm treating ApplicationService like a black box; I'm just testing return values!


So this is where my understanding breaks down. How would I go about black-box testing without mocks? It seems like I have to write white box tests for everything, which seems to create brittle tests that have to change as soon as as requirements change down the chain. At that point, why am I even separating functionality into classes and layers?

The example in this question is very simple but I hope it gets the point across; one could imagine an example with more layers where the test complexity would only grow.

  • Can you please provide by which definition of "tight coupling" are you measuring your examples? Reading this question, I feel there is big difference in what I would define as "tight coupling" and your usage of the term. – Euphoric Feb 11 at 11:01
  • In the same way you apply good development practices to your software (to avoid maintenance headaches), you should do the same with your tests, to decrease the amount of fragile tests (tests that break for the smallest change in the software/requirements). Other than that, I see no reasons to get rid of the mocking; it's meant to help you simulating whatever scenario you need for the testing, and you can do black box unit tests with that, I don't see how is that a problem. Take a look at this: blog.cleancoder.com/uncle-bob/2017/03/03/… – Emerson Cardoso Feb 11 at 11:05
  • @Euphoric You're right, I may not be using it in the defined sense. What I mean is that if ApplicationService changes in which instances in returns false, then the ApplicationController tests need to change as well. It seems to me, in the example I provided, that such is the case when not using mocks. Is that clearer? – Jokab Feb 11 at 12:55
2

It is correct that the test result depends on the "real" ApplicationService class if you use it in the test. That's exactly the problem that mocks are trying to solve.

If mocking creates brittle tests that need to be changed all the time, you are using mocks in the wrong places. In your example, the opposite statement is true: When using a mock, the test must be changed if ApplicationController changes. However, when using a concrete ApplicationService, the test must be changed if ApplicationController or ApplicationService changes.

|improve this answer|||||
1

What is mocking?

Mocking is the process of replacing classes which our subject-under-test depends on with implementations suitable for testing

When is mocking successful?

Mocking is successful when we can say we have created a test with minimal cost that effectively tests the subject-under-test

Corollary

We need to be very clear what the subject-under-test is. For example, when unit testing this ApplicationController, we definitely do not want or need to test the user login with it. We should test the user login in the user login component unit tests.

Other kinds of tests have different goals. You should have a test that that can test systems. However it is uncommon to mock these tests on the code level. Other patterns, for example fake API servers, may be more useful.

Why does mocking have a bad reputation?

Mocking becomess necessary when we have dependencies. It would be preferable not to have these dependencies in the first place.

|improve this answer|||||
  • 1
    "Mocking becomess necessary when we have dependencies." - Indeed. The amount and complexity of your mocks is a very obvious and painful reminder of the coupling of your system. – Jörg W Mittag Feb 11 at 11:13
  • 1
    "Mocking becomess necessary when we have dependencies." I would recommend to use clearer vocabulary. Mocking != Test Doubles . martinfowler.com/bliki/TestDouble.html First OP's code snippets is an example of in-memory database fake, not mock. – Euphoric Feb 11 at 11:15
  • Should I not have dependencies? Are you suggesting that layering into Controller/Service/Repository is a bad pattern? If so, what do you suggest instead? – Jokab Feb 11 at 12:52
  • Many dependencies seen in legacy systems are avoidable. So yes you should not have them if at all possible, – Martin K Feb 12 at 7:30
1

If you use a mother (fixture factory) to handle the creation of an authorized and unauthorized user ? Then it would be centralized in one place for your tests and there is no coupling I believe.

https://martinfowler.com/articles/mocksArentStubs.html

The mock is more coupled because in this kind of situation we tell the test how the logic is performed inside the SUT (the exact method on the service we call). It is more fragile regarding refactoring if you want to change the implementation.

In the other hand I think a controller and a service are in 2 separate layers and the interaction is supposed to be stable. Maybe you assert only delegation in controller and here a service stub (authorizedStub and unauthorizedStub) is a good thing.

https://blog.cleancoder.com/uncle-bob/2014/01/27/TheChickenOrTheRoad.html

I like to reduce unit tests (inside a layer) to be more free, it's effective but there is always something that goes wrong anyway lol.

<> https://mobile.twitter.com/unclebobmartin/status/943552605950750721?lang=fr

Look at this comparison between 2 styles of tdd : https://codurance.com/2015/05/12/does-tdd-lead-to-good-design/

The is no silver bullet like they say.

That is my vision at the moment, I hope it's useful.

|improve this answer|||||
0

Lets see to what each of the example tests is coupled.

With mocks:

public void LoginUser_UserNotActive_ReturnsUnauthorized()
{
    var serviceMock = Substitute.For<ApplicationService>();
    serviceMock.LoginUser(Arg.Any<string>()).Returns(false);
    var sut = new ApplicationController(serviceMock);
    var result = sut.LoginUser("someId");
    Assert.IsType<UnauthorizedResult>(result);
}
  • Coupled to ApplicationService
  • Coupled to ApplicationService.LoginUser method, that has String parameter and returns boolean
  • Coupled to when ApplicationService.LoginUser is called, return false
  • (assuming the test is actually complete) that ApplicationService.LoginUser is only called once and no other method is called
  • The above couples test to how controller implements it's behavior

Without mocks:

[Fact]
public void LoginUser_UserNotActive_ReturnsUnauthorized()
{
    var sut = new ApplicationController();
    _dbContext.Add(new User { Id = "123", Active = false });
    var result = sut.LoginUser("123");
    Assert.IsType<UnauthorizedResult>(result);
}
  • No structural coupling, the test doesn't care how controller implements it's behavior
  • Coupled to User existing in database with specific Id

Really, I would say the mock version to be much more coupled to ApplicationController implementation than the non-mock, database-faked version.

|improve this answer|||||
  • High-level integrated tests for covering components with external dependencies, unit tests for all business logic. Forces you to isolate your business logic from your dependencies. Simple DI (stays high in the call stack, only injects external dependencies). Keeps your domain pure. Everyone wins. – Ant P Feb 11 at 11:58
  • 1
    In your example, you introduce flakiness on the second test. Errors might happen since you're now testing actually using the application's runtime. Depending on your scenario, this might or might not be good (I personally would want to be able to rely on a test result). – Emerson Cardoso Feb 11 at 12:43
  • I realise my question is a bit rambling, so it may not be clear what my own current position is. To me it feels like the example without mocks is more coupled since if the requirement that the user must be active is removed, both tests must change. Furthermore, if I test UserNotActive, must I not also test UserDoesNotExist? Why do I test one and not the other? If I do decide to test both conditions for returning false, I'm basically testing the implementation, no? – Jokab Feb 11 at 12:50
  • 1
    "Really, I would say the mock version to be much more coupled to ApplicationController implementation than the non-mock, database-faked version." In fact you only hid this coupling from your test and this something really bad. – Timothy Truckle Feb 11 at 21:36
  • 1
    A dependency is not an implementation detail. – Timothy Truckle Feb 12 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.