Is microservices architecture a good candidate for a pipeline?

Question

I have a monolithic application which can be divided into multiple steps and different steps have variable scaling requirement, however the input of the next step is the output of the current step. Currently, we are not able to handle the no of requests to our server and we need to have more servers/load balancer/etc. We are also thinking to re-architect our application. If we create separate services (for those steps) and deploy them as containerized application using Docker & Kubernetes on cloud and use some distributed message broker (Queue) to pass the results from previous step service to next step service, we would be implementing a sort of microservices architecture. The only concern which I feel is that if Service1 container instance and service2 container instances are in different servers/hosts, then there would be some network latency which would be multiplied by the no. of requests.

So based upon my understanding the microservices architecture is not a good candidate for pipeline kind of requirement, if we are looking for real time performance. It will be better to keep those step based services in the same server and may be control the amount of resources which can be used by those services i.e. allocating more resources to service which needs them more and then we can auto scale the whole server based upon load. We can have in-memory queues between those services. Do we have a software which can help in dynamically allocating more resources to a service if the no. of items in their queue is high?

Answer 1

I have a monolithic application which can be divided into multiple steps and different steps have variable scaling requirement

This has nothing to do with a state machine. This is a pipeline.

You probably should break it into microservices so you can scale per logical component. Why do you need a state machine? Do calls affect subsequent calls? Generally back-end calls are designed to NOT change state of other backend calls. If there is only a little state per call, then just pass it along the microservice pipeline as parameters.

If you really need a truly global state, a microservice is probably the wrong answer.

EDIT in response to comments

In the comments, you say step 2 acts on step 1 (and step 3 acts on step 2). This is a pipeline. If package A goes into step 1 -> Spits out Package A1, package A1 is fed into step 2, becomes package A2. If this is the flow, then I don't see why you need a state machine. You need to dynamically scale "step containers".

The only concern which I feel is that if Service1 container instance and service2 container instances are in different servers/hosts, then there would be some network latency which would be multiplied by the no. of requests.

If you're using Google Cloud all the physical machines will likely be in one very well connected data center. In my experience, latency inside the data center has never been an issue with either Google Cloud or AWS.

So based upon my understanding the microservices architecture is not a good candidate for state machines, if we are looking for real time performance.

I don't understand the fixation on state machines. Why do you need one. In a cloud environment, a state machine is a way of sharing global state between instances. The way the question reads now I don't think you need one.

resources which can be used by those services i.e. allocating more resources to service which needs them more and then we can auto scale the whole server based upon load.

So you're essentially redoing Google Cloud's work for them. Google Cloud offers something called a pipeline, which sounds like exactly what you need. The point of both Google Cloud and AWS is to move away from "I bought the box" to "I bought computing power". Google Cloud will auto-scale the pipeline, bringing more instances of each microservice online as it's needed, and deleting it when load tapers off. Don't redo this - you're probably not as good at this as Google or Amazon. Let them handle it - that's what you're paying for!

Answer 2

Separating them into individual services so that they can scale independently - good idea. For real-time responses, adding requests to the message queue will not help - you are right.

Network latency would be there if you are making a call to another service regardless of the queue, you have to ensure the services are in close proximity, have a circuit breaker while making downstream calls to ensure responsiveness is not taking a hit.

keep those state services in the same server and maybe control the amount of resources which can be used by those services

This is precisely the reason to move to Microservices and scale horizontally - managing resources will get extremely hard on the single machine. Scaling vertically has limitations, in terms of cost, in terms of ease of scaling too. If your scale is not much, and don't see it increasing by much - you can consider this though, it's a lot simple but beware of its limitations.

Just a thought - You can think of deploying your services as a side-car. This will help reduce latency between services, provide different resource configurations, scale horizontally easily as well.

Answer 3

As always: It depends on what your load/task is.

The trap I'd see in your situation is, if you have a lot of small step working over a small amount of data, you may end up with the cost of transfering thing from one place to another being too much. (I remember reading an article about super computer, in which engineer said it was often cheaper to run a calculation localy than import a cached result from another super computer)