Idiomatic internal architecture of Go microservices

Question

For learning purposes, i'm trying to implement a small project using (buzzword warning!) microservices. There are plenty of resources online talking about the 'macro' microservice world -integration, orchestration, service discovery, health checking, topology, etc., however I've found that there is little to no information about internal implementations.

The best one i've stumbled across so far is this diagram from Martin Fowler's site. He details one possible internal configuration utilizing a facade pattern with a service encapsulating a domain and repository.

However, after having a hard time trying to implement these patterns myself, and after looking at various other opinions, apparently these kinds of abstractions don't fit in well with Go's general philosophy of taking the most straightforward approach?

My initial thought is that most of the time there is simply no need for these kinds of abstractions given the nature of microservices, assuming they have been properly partitioned into specific and limited bounded contexts; if a service is just a few hundred loc, then you're better off rewriting a tightly coupled ball of mud then designing the perfect abstractions just for the sake of testing/reusability.

Broadly speaking, is this the case? Are there other patterns that might better fit the Go paradigm for designing the internals of a microservice? I'm not necessarily "indoctrinated" into any particular dogma, but I am interested in reading the various viewpoints.

Answer 1

The external microservices are exactly that: external. You should consider them as a dependency with which you interact through HTTP and which shouldn't be trusted.

• The fact that it shouldn't be trusted creates a first set of difficulties. Even if you have a well-specified interface, you shouldn't assume that the other side will obey to it. It might start responding weird things, but more probably it could bring a specific behavior such as start blocking requests when they are done too frequently.

• The fact that HTTP is used creates additional complexity. What if the DNS server is down? What if the network connection dropped in the middle of the transfer?

This forces you to think about handling exceptional cases you don't usually handle within the process itself:

• Handling and reporting of errors related to the dependency,
• Failover, when relevant,
• Handling of timeouts,
• Continuity of service if the dependency is not responding any longer,
• etc.

(A good book about those aspects is Release It! by Michael T. Nygard.)

This means that you should consider an architecture which takes in account the unreliability of the dependency. Essentially, this means that:

• The layer which is in charge of the microservice should make it safe for your application to use the microservice. It should handle the timeouts, the issues with the network and the changes within the interface as well as possible. Unfortunately, it will end up being a leaky abstraction, but it's still easier to handle service-related issues at this level than let the exceptions flow freely within the core layers of the application.

  In the same way, the layer which handles the database should, ideally, encompass all the database-related logic. If you suddenly need to replace Oracle by PostgreSQL, the impact on your code should be strictly limited to this layer.

• Each microservice client should be in isolation from other microservices. It's not one big class which handles the calls to twenty microservices, but rather twenty independent parts which handle a microservice each; they can still share (inherit, whatever) the same code when needed to avoid code duplication.

  Similarly, if your application relies on MongoDB and Redis, you don't put one big class which handles both. You have specific providers which handle MongoDB, and other providers which deal with Redis. Replacing Redis by some other data structure store means that the code related to MongoDB will remain unchanged.

• The app may need to handle the case where the microservice is not available any longer. For instance, if you're building an e-commerce website, the fact that the microservice which handles user comments of products is not responding shouldn't prevent you from showing the home page of the site or keep selling products.

  Similarly, the loss of LDAP may be an issue for the parts of an e-commerce website which actually require authentication, but the application can still show its home page to guests. If losing LDAP is a likely scenario (I hope it's never the case), changing the e-commerce website so that purchases could still be made without LDAP could be a solution.

• Replacing a microservice by its newer version should affect only a small part of your application, and shouldn't propagate to every layer. New versions of microservices will be released, and older ones will become obsolete, so your life will be much easier if you design your code from the beginning to be able to swap a microservice client by another one easily.

  In the same way, changes to Twilio, Facebook or Google Maps API shouldn't cause you too much pain. You may need to patch a few classes which do exactly that—deal with those APIs and present them to your app as an abstraction—but the changes are not expected to propagate through the layers of your application.