-1

I'm writing a class that acts as the interface to a server. The interface exposes a way to send messages to the server and pass messages back to the client through a callback. Implementations should convert messages into the appropriate format for the server and handle the sending logic. Essentially the API might look something like:

class Server:
    def __init__(self, callback)  # callback when the server sends a message

    def send(message):
        # format and send a message to the server

To me this is a clean API and means that any details about how the Server actually connects are nicely encapsulated and hidden away from the client of this code.

However, trying to write tests for this design turns out to be a bit of a nightmare - I have no way to control whether a message is sent to an actual server, which obviously I do not want in a unit test. Of course there are hacks around this, such as using patches, but this makes the tests dependent on implementation. Even if I passed in a mock URL to the constructor and spun up a mock server, this would be a lot of effort and outside the realm of unit testing.

The alternative that I see time and time again, is to use some form of dependency injection to pass the actual connection object, so something like:

class Server:
    def __init__(self, callback, connection)

    def send(message)

Now this allows us to mock the connection in tests and control what is sent back at the unit test level. My issue with this is that I think it breaks the encapsulation that the first choice gave me just for the sake of being able to write tests, and from the client's point of view actually feels (to me) like a worse design. All this really seems to do is actually force the client to know about the implementation details of the connection.

For example, if I want to retrieve data from ServerA and ServerB, as a client, I do not care how the data is retrieved from these servers. It might be that ServerA only transmits the data using REST and ServerB uses websockets. In option 1, this is hidden away from me, but in option 2 I have to be aware of this. And if I then have a ServerC that perhaps uses two different connections to retrieve different types of data, it seems my design breaks.


I know another argument may be that this is the 'data access layer', and so I should avoid putting logic in these classes and therefore I do not need to unit test them.

If we suppose that the client knowing about the implementation details of the connection is not a bad design, I can see an option here might be to do something like this:

class Websocket(Connection)
    def __init__(self, url)

class REST(Connection)
    def __init__(self, url)

class Server:
    def __init__(self, callback, connection):
        # Take a connection object, the server class handles formatting a message
        # and passing it on to the connection object

Then based on the connection to each server, we can pass the relevant Connection object (Although this still doesn't seem to address the problem with ServerC). But, I still am unsure about how I could unit test any of the Connection classes. It seems hard to imagine they would contain no logic whatsoever, e.g. at a minimum I would expect some interacting with a third party library, and to truly test this presumably I would need to run some kind of server to verify that a connection happens, which also seems quite difficult.

What is the correct way to unit test this? Is it perhaps more pragmatic to simply go with the first approach and rely on integration tests, or does the second approach make sense, without overengineering/substantially increasing complexity?

3
  • You don't use dependency injection like that. Any real-life system would use DI on the message broker, so that it can be easily tested, and then expose a Façade over the broker to the client programmer. The user doesn't have to know about the implementation, and the wrapper is so trivial that it doesn't need extensive testing. Feb 8, 2022 at 17:36
  • @KilianFoth thanks for your reply - could you clarify a bit further please? It sounds like you're saying the second route is the way to go, but why would I need a facade on top of the message broker? To hide the DI on the message broker? Feb 8, 2022 at 17:41
  • 2
    Yes, you said exposing the injected connection breaks encapsulation and burdens the caller with unnecessary detail. And you're quite right! So instead of exposing it, you should hide it. But that doesn't mean you can't use DI as well. You end up with three classes instead of one, but each serves its single purpose better than the original Server would. Feb 8, 2022 at 18:23

1 Answer 1

3

To me this is a clean API and means that any details about how the Server actually connects is nicely encapsulated and hidden away from the client of this code.

You can certainly hide the details of the implementation behind a module interface, but be careful not to hide your edge cases. Connections to the network can fail. Connections to the network can time out. Think carefully about whether the client code needs to be able to handle these conditions, and how you want to communicate them.


For testing the clients of this interface, you introduce a substitute implementation that is completely deterministic, and arrange the design of your clients so that the substitute can be readily introduced in place of your "real" client.

Generally, that means passing either an instance of Server or an instance of Server.send to your client somehow - it might be a constructor argument, it might be a method argument, it might be a property override. Lots of different options here.

Note the big advantage of using a substitute is that you can design substitutes that can deterministically produce the error conditions we were talking about before.


For testing Server itself, there are a couple of possibilities.

Assuming for the moment that you aren't implementing your own network code, but are instead using some general purpose library to execute the side effects, then a simple answer is to play the same game again: create a substitutable abstraction of the network library itself, and test your Server component using the substitute. For real live communication with a remote server, you swap in the real code.

In other words, you've got a "turtles all the way down" approach to introducing substitutable implementations UNTIL you reach a point where the real code is "so simple that there are obviously no deficiencies".


My issue with this is that I think it breaks the encapsulation that the first choice gave me just for the sake of being able to write tests, and from the client's point of view actually feels (to me) like a worse design. All this really seems to do is actually force the client to know about the implementation details of the connection.

Not a bad instinct, but keep digging.

An important idea is this: the code that invokes Server.__init__ and the code that invokes Server.send are not the same code.

The Composition Root is responsible for wiring things together, and you may or may not be able to insulate that code from initialization complexity. But once the initialization is finished and the logic is running... all of that code is "structurally agnostic".

You might also benefit from looking into the Doctrine of Useful Objects, where you build into your design the affordances to control the dependencies of your components.

It's all trade offs - you are unlikely to find a design that gives you everything, but you can normally achieve a design that gives you what you prioritize.


Note that "unit tests" are weird birds - they usually have at least some of the character of a composition root because the test subject has to come from somewhere. If you squint a bit at the "arrange" tasks in "arrange/act/assert", you'll often see lots of initialization there.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.