0

I am writing a REST API for use by the front-end web and mobile frameworks. Naturally, I write unit tests for each endpoint, which makes developing, evolving, and maintaining the API easier. However, currently, my unit tests are not atomic since they save and load certain variables into a persistent file.

For example, let's say I have endpoints for user creation, reading, updating as well as login/logout endpoints. Each endpoint has its own corresponding class which contains several unit tests that test a successful response as well as several failing responses. So, to create a user, the request to user/create has to specify an unused email, and do that I keep a counter which I append to the email address. Furthermore, I use this persistent counter in my user/read unit tests to compare against the expected values such as email. So this obviously couples the two together and makes them not atomic. So what is the better way of organizing the tests in this case? An easy answer would be to just perform the create/read/update requests within a single test but that seems to me to go against the principles of modularity and separation of concerns. And it still doesn't solve the unique email problem.

Another example would be testing authentication and other endpoints that require an authenticated user. So to perform those test that require authentication, such as a request to user/update I would need to have an API token ready for it, which means I must have run the auth/login request to get the API token and saved it somewhere to be used by subsequent tests. Therefore, it obviously introduces the same problems as the above-mentioned example, if not more.

Hence the current organization necessitates a rigid sequence of tests, which goes against the principle of atomicity, which in turn renders features like parallel testing impossible. Therefore, I am keen to hear what is the right way of unit testing REST API to facilitate atomicity?

0

3 Answers 3

4

A full test suite should contain three levels of tests:

  1. Unit tests
  2. Integration tests
  3. End-to-end tests

Automated testing for REST APIs is typically done with integration tests, exactly for the reasons mentioned in the question. The unit test rules do not apply for integration tests.

2

As for automatic testing, what you are missing is the capability to either

a. Regenerate the datastore in a very specific state during the testing phase, so you can provide each test with the required data and the required state. If we were speaking about databases, I would suggest implementing in-memory databases alongside with some sort of change logger.

b. Mock dependencies.

What you are looking for is named determinism. For you to be able to execute tests in any order, anywhere, anytime and always obtain the same results you need the capacity to force the data that makes the business logic to behave in one or another way.

Another example would be testing authentication and other endpoints that require an authenticated user. So to perform those test that require authentication, such as a request to user/update I would need to have an API token ready for it, which means I must have run the auth/login request to get the API token and saved it somewhere to be used by subsequent tests. Therefore, it obviously introduces the same problems as the above-mentioned example, if not more.

Hell no.

You can create a default token for testing. One which never expires and bound to a non-production user. For example, I have tests that help me out to create and validate JWT. Those tests are never executed as part of the testing plan. They are tools somehow.

I turn these tokens into constants in my testing code. If I have different profiles, I generate different tokens, each of which is bound to a non-real user with the expected role for the test.

Hence the current organization necessitates a rigid sequence of tests, which goes against the principle of atomicity, which in turn renders features like parallel testing impossible.

Not necessarily. Say you achieved determinism in your automatic testing. You still can deploy a staging environment for E2E or Loading testing where the kind of tests you have described fit well.

2
  • Creating a default token for testing seems to be the easiest approach. That was of course of one of the first solutions that came to my mind, but I was not sure if this was a proper approach, since that means the same user would be tested every time, as opposed to various users. But now I realize that this was not necessary and one pre-defined user should be fine and would greatly simplify testing. Commented Oct 27, 2020 at 14:10
  • You can simulate real usage with many and different users. But that's not the goal of unit tests. I would address such tests to a different stage. For example to E2E or load tests. Load tests, for example, are addressed to stress the system and that's a very good place where to set all sort of users and use-cases. Units test are test addressed to validate the reliability of a small portion of code. Like would you to test the resistance of a nail. You would not build the entire Empire State Building to demonstrate the nail is reliable enough.
    – Laiv
    Commented Oct 27, 2020 at 15:06
2

my unit tests are not atomic since they save and load certain variables into a persistent file

Persistency is orthogonal to unit testing. Unit tests should not be using any kind of persisted or shared state.

The simplest example here is your car. Suppose it doesn't start. What's broken? Well, you can't know that. It could any of the many parts in your car.
But what we can do is take the car completely apart, test each component individually, and then see which component fails its test. That's most likely the broken part.

Let's use the carburetor as an example. Its job is to take in fuel and air, and output a fuel/air mixture. The goal of our test is to see if it outputs the correct mixture.

In your case (as described in the question), you've attached the carburetor to the car's fuel tank and air intake, and are now testing to see if the correct mixture comes out. If it does, then the carburetor is likely not the reason why the car won't start.

But more importantly, if it doesn't output the correct mixture, does that prove that the carburetor is broken? No. Because maybe your car's air intake is broken, or your fuel tank. That would also cause the carburetor to not output the correct mixture, since it's not receiving the correct input.

This is the problem in your testing strategy. Your (supposed) unit test isn't testing a unit (i.e. one component, the carburetor), but rather a composition (i.e. multiple components, the carburetor, intake and tank). So when that test fails, any of the components could have caused it.

So how do you properly unit test?

Going back to the carburetor example, what we should have done is connected the carburetor to a "fake" fuel and air supply line. One where we know for a fact that it is working correctly.
When we have such a "fake" input, we could even use a different liquid (e.g. water instead of fuel) and different gas (e.g. methane instead of air) and see if the carburetor outputs the correct water/methane mixture.

This is what we call a mock. The mock essentially gives us full and direct control over the input, so that we can judge the component based on its output.

Let's look at your code, e.g. the user update logic. Let's say that you have two relevant classes here:

  • UserManager, which contains the user update logic
  • UserRepository, which the user manager depends on, and handles the database interaction

A simple example:

public class UserManager
{
    private readonly IUserRepository _repository;

    public UserManager(IUserRepository repository)
    {
        _repository = repository;
    }

    public bool UpdateUser(int id, string newName)
    {
        var existingUser = _repository.GetById(id);

        if(existingUser == null)
            return false;

        existingUser.Name = newName;
        _repository.Update(existingUser);

        return true;
    }
}

Returning booleans isn't really the best approach, but it's a straightforward if oversimplified example. true means the user was updated, false means there was no existing user.

We're trying to test the user update logic, so we should unit test the UserManager. Since a unit test means that you test one component, that means that we're going to be using a real UserManager, and everything else will be fake. So we must create a "fake" user repository, one where we can directly control exactly what it does.

Note: I'm using NSubstitute and FluentAssertions syntax here because I find it very readable to understand what's going on. You could roll your own code or use other libraries, but I'd personally recommend these two as they're quite beginner-friendly.

var mockedUserRepo = Substitute.For<IUserRepository>();

This creates a fake object. Think of it like a double agent. It looks and handles like a user repository, but it's actually doing our secret bidding. We still need to tell it what that bidding is, though.

In this case, we want to test the "happy" path, i.e. an actual update to an existing user. This means that there must already be a user that exists in the database. So the repository must pretend like such a user exists.

var fakeUser = new User() { Id = 1, Name = "John Doe" };

mockedUserRepo.GetById(1).Returns(fakeUser);

We've now instructed our double agent. If someone calls the GetById method and passes 1 as a parameter, the double agent will return fakeUser as the return value.

We're now ready to run our unit test! We create a real manager, but we give it a fake user repo, our double agent:

var userManager = new UserManager(mockedUserRepo);

Now we call the method we want to test and store the result:

var result = userManager.UpdateUser(1, "new name!");

And now we check to see if the update was executed, which in the case of our oversimplified example means checking the boolean return value.

result.Should().BeTrue(); 

However, it would also be good if we could confirm that the repository's update method was called and contained the new user name (not the old one). Luckily, our double agent kept a record of everything that's been done to it, so we can ask the double agent to confirm that a repository update was attempted.

mockedUserRepo
    .Received()
    .Update(Arg.Is<User>(user => user.Name == "new name!"));

What we're asking here is if our double agent had its Update method called with a User parameter whose name was equal to "new name!".

Important: We are not trying to establish if the user was actually updated in the database. We're trying to establish whether the user manager instructed the user repository to update it. We can assume that (a) if the user manager did indeed instruct the repository to update and (b) the repository's update unit tests (not discussed here) all pass, that means that the user would have been updated when a real manager talks to a real repository.

Now, we have a full unit test, and we didn't even need a real repository, let alone a persistent data store!

But we should test all paths, not just the happy one.

For completeness' sake, we also want to write a second unit test, where there is no existing user, to confirm that the user manager does not try to update a user that does not exist.

This is similar to the previous test, but with some alterations:

var mockedUserRepo = Substitute.For<IUserRepository>();
mockedUserRepo.GetById(1).Returns(null);

var result = userManager.UpdateUser(1, "new name!");          // alteration

result.Should().BeFalse();                                    // alteration

mockedUserRepo
    .DidNotReceive()                                          // alteration
    .Update(Arg.Is<User>(user => user.Name == "new name!"));

Note the alterations:

  • The mocked repo returns null, as if the user doesn't exist.
  • result should be false
  • The mocked repo must not have received a call to its Update method.

I hope the above step-by-step explanation highlighted the reasoning and approach to mocking your initial data values, so that the only "real" code you rely on is the code that's being tested.

For each and every piece of logic you described in your question, you should be able to set up the initial mocked data state without relying on other real code, i.e. you don't rely on your user creation logic to test your user update logic, and you don't rely on your actual access token to test if the user is authorized to perform an action.

Instead, you simply pretend that the situation is how you want it to be, and then test the specific unit that's being tested, nothing more.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.