Here's a simple scenario: a REST API of which you launch multiple, load-balanced replicas of the same service with Gunicorn. Unit and integration tests are run in single-instance cases, but how can you run integration tests that include all instances so you check that they have a single source of truth?

I can think of several ways but I don't know if there's a better one in Python (or in other languages). Maybe I just miss the vocabulary to search for it. Here's what I think

  • Launching the several instances of the server from some external program, and use API requests from the test to check that.

  • Use Gunicorn as a module, write a script that works in the same way as the command you used, and run integration tests on that one.

Is there any better way to do this?

  • Are the multiple instances supposed to be replicas? (for load balancing) or does each instance correspond to 1 resource (endpoint)? In other words, are you asking how to validate if all the instances share the same configuration?
    – Laiv
    Commented Jan 7, 2020 at 13:10
  • @Laiv they are replicas. I'll write a clarification for that.
    – jjmerelo
    Commented Jan 7, 2020 at 13:42
  • This sounds like an operations thing. Since operations has to be done independently for every environment, testing this on your test environment gives you 0 guarantees about your production environment. So to clarify: where do you want to run these tests? Commented Jan 7, 2020 at 14:33
  • If you are testing through a load balancer, you are not testing instances, you are testing load or routing. So, @JoryGeerts is hitting the nail, where do you want to run these checks and why? Bear in mind that, behind the LB there could be (literally) anything and you should not allow others to guess what lays beneath. A different scenario is if you want to check settings from behind, instance by instance.
    – Laiv
    Commented Jan 7, 2020 at 15:27
  • @JoryGeerts if possible, production, but in any case in any environment that uses some kind of load-balancing (and web-serving) front-end and an already-tested-singly back-end.
    – jjmerelo
    Commented Jan 7, 2020 at 17:13

1 Answer 1


In these senarios I want to test two things

  1. Do my tests pass for each individual server

  2. Do my tests pass for the load balanced endpoint

In order to test individual servers I need to be able to address them directly. This can be done by going direct to each box, or by providing a routing parameter to the LB which bypasses the nomal host selection.

I can check config properties such as your "are they all using the same db?" test by giving each a healthcheck endpoint which exposes configuration.

So im running the same set of tests I used in dev and test envrionments, I just have more targets to run them against.

  • OK, I can imagine that can be done, but is there any tool that does that automatically? In Python or other language?
    – jjmerelo
    Commented Jan 7, 2020 at 17:12
  • any test runner?
    – Ewan
    Commented Jan 7, 2020 at 18:20
  • if that's what they're called, yes.
    – jjmerelo
    Commented Jan 7, 2020 at 18:30
  • wait. so how are you currently running your integration tests?
    – Ewan
    Commented Jan 7, 2020 at 18:35
  • this question was purely theoretical, but for the sake of clarity, let's say the API includes integration with a database, and is tested anyway on a single replica, without starting (via gunicorn or any other front-end/load balancer) several replicas at the same time. At any rate, single replica, no gunicorn.
    – jjmerelo
    Commented Jan 7, 2020 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.