Decoupling microservices with gRPC

Question

I'm setting up a microservices architecture, and am confused about how gRPC can loosely-couple services (compared to a pub-sub message service like Kafka). Doesn't the request go directly to the server, and not through a pub/sub system? While gRPC supports asynchronous requests, should I still use pub/sub as a buffer between services to scale them independently?

Answer 1

gRPC and queues / brokers are not mutually exclusive. We could choose one, both or none. We could also use an SMTP server as a queue, or a plain text file.

In other words, they complement each other because they satisfy different purposes.

Decoupling

Think in a pipeline where stages are Services.

[Service A] < ----- > [Servicie B]

^{Arrows denote cardinality and dependency.}

Both services are client and server at the same time. A is client and server of B. And vice-versa. Regardless the protocol, the IPC here involves services being aware of each other.

Now, set a queue or broker in-between

[Process A]  <----- > [Queue|Broker] <------> [Process B]

^{Arrows denote cardinality or dependency.}

Queues and brokers break the cardinality between services, hence the dependency. They no longer need to be aware of each other. We could drop any of the services or change them so that the other remains unaware of these thingse.

Adopting features

Queues and brokers are usually designed to be performant/reliable/consistent/scalable/resilient I/O and we can provide our architecture with these features by implementing one or another. For example, gRPC is a transport mechanism for request/response and (non-persistent) streaming use cases. If we can not afford missing messages, we probably will have to look for alternatives or complementary architectural components. Queues and brokers might do the job. Persistence is not the only feature to provide our IPC with. To mention a few

• Message consistency and persistence
• Catching and batching
• Error handling
• Failover
• Retry
• Scalability
• Modularity or compartmentality (having dedicated brokers or queues)
• Compatibility with several protocols out-of-the-box
• Traffic traceability
• Traffic monitoring
• Tooling

Related links

I found this blog post which dives a little more into the subject.

Worth a mention the Nginx blog post about Microservice and IPC implementations.

Answer 2

gRPC can do asynchronous requests, but it is still an RPC pattern and so that is mostly for when you don't care about the answer, not for implementing message (sub/pub) systems.

What I do is, if I need RPC I use gRPC for the communication and ProtocolBuffers to serialize the data, and then when I need messaging I use AMQP, also with ProtocolBuffers.

It makes sense to have the gRPC buffer the data through a messaging system if you need to scale it, have routing between servers, or if you want features like offline messages and also message receipt notification.

My advice is to diagram the communications scenarios on paper, and it should be clear which type of service you need.

Also, note that using gRPC is mostly turn-key and AMQP isn't; generally messaging systems will require you to write a lot of code. If you're not really scaling it very much then it might be an excessive amount of work. Also, I think it is rare that you would want both; the RPC pattern is mostly a subset of the messaging pattern. If you're going to outgrow RPC, use messaging from the start IMO.

Answer 3

Yes the request will go directly to the server.

gRPC allows a developer to generate interfaces for the various services. Clients only need to depend on the interface and not an implementation.

typically interfaces can be designed to be backwards compatible. So when a service changes, new functionality is added to the interface and no functionality removed. Therefore old clients can talk to newer versions of the interface while new clients can access the new features. Once all clients are upgraded then old deprecated parts of the interface can be removed.