10

When writing unit tests, it's common to use fixtures: little testable data, so we can say: 1. Get all clients should include Willy Wonka. 2. Delete client 3, and now get clients should not include Willy Wonka anymore.

That's fine for unit tests. Use setup/teardown to re-load the fixtures or rollback the transaction. So testing creates, updates, and deletes are done inside a transaction. The new temporary data lasts just for the length of that test, then is reset.

But what about when we've separated the REST server from the REST client?

We want to make sure our REST client is not just reading correctly, but creating, updating, and deleting correctly.

I haven't been able to find any examples or suggestions for how to do this against a remote test REST server.

Assuming I've got a test REST server serving only fixtures. The whole stateless nature of HTTP means it'd be hard to send a "BEGIN TRANSACTION" and "ROLLBACK TRANSACTION" or "RELOAD FIXTURES" type of message, right?

I can't be the first to want to do this, so I have a feeling I need a different way of thinking about this.

Any suggestions?

  • Perhaps, since it is a test server you can have an endpoint that will reload the fixtures? – David Radcliffe Mar 9 '13 at 1:59
  • If your main problem is to bring your test server back into a predefined state, why don't you add some kind of special testing functions like "RELOAD TESTDATA" to your rest API for doing what you want? Of course, you should make sure that kind of API calls is not available in production. – Doc Brown Mar 9 '13 at 8:53
7

Software systems ideally have well-defined system boundaries and interfaces between them. REST services are good examples of this.

To that end, I would recommend against doing what you're trying to do.

Specifically:

We want to make sure our REST client is not just reading correctly, but creating, updating, and deleting correctly.

What I would suggest instead is:

  • Building tests for your REST client, to ensure that it behaves correctly, given specific input and output. Account for good (expected) and bad (unexpected) values.

  • Building tests for your REST service (if you control it, that is), to behave according to its intended function

  • Keep tests close to their problem domain, so they can help guide the design and development of what is important in that context

  • 3
    You dismiss the whole idea of integration tests here quite casually. I don't think this approach is informed from practice. – febeling Mar 9 '13 at 13:47
  • Thank you to all the helpful suggestions. Also by Twitter I got great suggestions to try Ruby gem "webmock" and similar to mock the REST API server response. I also agree with "febeling" that what I'm describing seems to be more of an integration test, so I'll look into that separately. Thanks again everyone. -- Derek – sivers Mar 10 '13 at 8:13
  • mocking an API is great way to solve the problem. But how do you make sure that mocked API == real API? – FrEaKmAn Jan 6 '15 at 13:54
4

Two angles to keep in mind here:

  • Are you testing your code or the plumbing? Presuming you are using a well know service and client stack you can probably safely presume their testers and thousands of users will generally ensure there isn't a fundamental bug in the underpinnings.
  • Why aren't your tests idempotent? Make a way to write non-production data or write to a different endpoint. Pick some predictable naming pattern. Pre-load the rest server DB before the tests. And there are probably a few more ways to make this happen -- method is really tactical and should depend on the nature of the app.
2

I think faking the REST server responses is the best way to test the client.

For Ruby, there's FakeWeb gem which you can use to emit fake responses - https://github.com/chrisk/fakeweb.

Also, in JavaScript you can use something like Sinon.JS, which gives you a fake server - http://sinonjs.org/docs/#fakeServer.

1

As others have said, if you're testing a client you don't need to go as far as creating, deleting, etc. on the server. A lot of the time you don't even need to mock a server at all. You really only need to make sure you are making the right requests and correctly handling the responses, Whether it's written in Ruby, Python, PHP or anything else, at some point your client is probably going to use a method from an HTTP library to make a request and it's enough to mock that method, check how it's called, and return a test result.

Take a hypothetical Python client that uses urllib2 for making requests. You probably have some method in the client, let's call it get(), that has a call to urllib2.Request() in it. You only really need to mock the call to your own class's get().

@patch('your.Client.get')
def test_with_mock(self, your_mock):
    your_mock.return_value({'some': 'json'})
    test_obj = your.Client.get_object(5)
    your_mock.assert_called_with('/the/correct/endpoint/5')

This very simplified example uses Python's Mock library to test a hypothetical your.Client class with a get_object() method that generates the correct url to get something from some API. To make the request the client calls its get() method with that url. Here, that method is mocked (your.Client.get is "patched" so that it is under the control of your_mock), and the test checks whether the right endpoint was requested.

The mocked method returns the configured JSON response (your_mock.return_value) which the client must handle and you would make further assertations to test that it handled the expected data in the expected way.

  • But how are you sure that when you are making the "right" request that it's an actual right (in production) request? Because if I understand your suggestion, if the API changes or breaks, your tests will still work. While in production it's totally different story. – FrEaKmAn Jan 6 '15 at 14:00
1

What you describe is an integration test scenario. These are usually a bit awkward to setup and tear down. It makes them slow to run and quite often brittle.

The approach with fixtures is just as awkward and clumsy, but it is the default way some frameworks go about it, e.g. Rails, and it is already supported. They need the abstract test case or something similar to prepare the database with fixtures. (Beware of unusual naming of test categories in Rails, the unit tests with DB fixtures are strictly speaking also integration tests.)

The way I would go about your scenario is to accept to have test-specific control over the API application state or its database. You can either have additional endpoints for setup and teardown, which are only present in the test environment. Or alternatively, you talk to the database (or whatever you're using) behind the back of your application / API.

If you feel this is too much (in the sense of undue) effort, then consider that the approach with fixtures for databases does just that: using additional, test-specific means to manipulate the database or application state.

I don't the think this discussion has to do with stateless nature of HTTP, though. HTTP is stateless, but the application is definitely not, in the majority of cases. It sounds a bit like you where looking for REST strictness. You could just as well have all resources be fully creatable, readable and deletable. In that case you could just do all setup and teardown through regular API means. This is often not done in practice, though, because you don't want to include certain operations from a business understanding of your application, at least not outside of the test environment.

1

Monkey Patch

At my work we do ATDD by using an exiting xUnit framework and monkey patching network calls between the client and server. In the same process space we load the client, monkey patch the network call to the top of the REST server stack code. All calls are then issued from the client like they would normally be, and the server code gets the requests exactly as they would normally appear.

Benefits

  • no having to synchronize with server startup (because there is no server)
  • leverage classic unit setup and teardown method to manage things like fixtures
  • ability to use mocks/stubs and other monkey patches for more fine grained control of the test
  • can be written using a xUnit framwork

Tradeoffs

  • doesn't expose multi-process interactions/issues (locking, resource starvation, etc)
  • doesn't expose multi-server issues (data serialization, clustering style)
  • doesn't expose network issues because that is simulated (access, timeout errors, etc)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.