2

Like in the title: how do you integration-test server failures that return 500 HTTP responses?

For example:

  1. There's a server with a /save endpoint that accepts files on POST and saves it into the server filesystem
  2. There's a library/dependency X which handles the file saving functionality somewhere deep on the other end of the application
  3. X can fail in a number of reasons, but for this example let's say it crashes because there's no space on the disk
  4. We would like to throw an HTTP 500 instead of making the process crash

What do you do in this situation?

  1. Do you mock X and actually throw an error in the test?
  2. Do you omit these kinds of tests and just handle errors somewhere globally?
  3. Do you just integration test the happy paths and ignore error cases and test them in unit tests?
  4. Something else?

And one question that maybe answers my whole struggle:

Is testing if every possible error is returning an HTTP 500 something you'd call overengineering or is it rather a good practice?

1
+50

Basically you have three things to test.

  1. Does the component throw the correct exception when you are out of disk space?

    This can be checked by mocking the dependency in a unit test and throwing the desired error.

  2. Does the hosting layer return all exceptions to the user as HTTP 500

    This can be checked by having any integration test which tests an error scenario. Possibly one of your other tests which you know will throw on certain input.

  3. Does the specific exception in question return a HTTP 500 error

    This is harder to test but you can still do it by injecting your mock into the full application. I would code the mock to return different exceptions depending on input to allow the same mock to test a variety of scenarios.

In practice I would skip 3 unless there was something weird about it. You are just testing your global error handling and can never cover all of the possible exceptions

  • Thanks for your answer! Unit testing the throwing component is pretty easy and it's something I always do. The only struggle that I have is how to test if this specific exception makes the app return HTTP 500 (on the integration side) and by making this I always create a lot of hacks to inject the mock (which takes a lot of time, that's why this answer has been created). Does it mean I'm overengineering a little bit? :) One more question to point no. 2: so is it enough to integration-test one error case that I expect to return HTTP 500 (by an error handler)? – sarneeh Aug 29 '18 at 9:16
  • 3
    ask yourself "what bit of code am I testing" if its your generic error handler, then any error will test it. If you have special handling for this specific error then you might need more tests – Ewan Aug 29 '18 at 9:21
  • You're right. That makes total sense. I guess I fall too much into my perfectionism :) Thank you! – sarneeh Aug 29 '18 at 9:26
0

In the end-to-end integration tests, I would focus on the happy paths and a few, easy to trigger, fault scenarios.

The more elaborate or hard to trigger fault scenarios should be covered in unit tests and component tests.

In the scenario you describe, there should be a test for

  • the app to verify the behavior when the server responds with code 500
  • the server to verify it responds with code 500 if there is an unexpected failure in library X
  • library X to verify it gives a proper error indication when the filesytem is full.

These are all unit/component level tests with most of the dependencies mocked or stubbed.

  • But this is impossible to do as an unit test. Unit test tests small units, one particular test subject. The only thing you should mock there are direct dependencies to isolate the unit under test from other stuff. As I mentioned before - X is somewhere deep in the project, and the server (framework stuff) is somewhere at the entry-level. You'd certainly not want to mock the whole way down to X. This is why I use integration tests - which mostly use the real code on it's path to test if the system integrates well. I'm not sure if you got my point :) – sarneeh Aug 26 '18 at 12:19
  • @sarneeh: In a larger system, you may want to test larger portions of the system in isolation from the rest, which is why I mentioned component tests alongside unit tests. A component would be the result of integrating several units (but not all of them). Note that a component test (that verifies that a component by itself works correctly) is different from an integration test (which verifies that different units or components work correctly together). – Bart van Ingen Schenau Aug 26 '18 at 17:37
  • I guess I'd have to see some examples because it's hard to me to understand the differences and how does it look like in practice. Do you know any resources on this topic that could help me a little bit to pick it up more easily? – sarneeh Aug 26 '18 at 23:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.