0

I have a monolithic application which can be divided into multiple steps and different steps have variable scaling requirement, however the input of the next step is the output of the current step. Currently, we are not able to handle the no of requests to our server and we need to have more servers/load balancer/etc. We are also thinking to re-architect our application. If we create separate services (for those steps) and deploy them as containerized application using Docker & Kubernetes on cloud and use some distributed message broker (Queue) to pass the results from previous step service to next step service, we would be implementing a sort of microservices architecture. The only concern which I feel is that if Service1 container instance and service2 container instances are in different servers/hosts, then there would be some network latency which would be multiplied by the no. of requests.

So based upon my understanding the microservices architecture is not a good candidate for pipeline kind of requirement, if we are looking for real time performance. It will be better to keep those step based services in the same server and may be control the amount of resources which can be used by those services i.e. allocating more resources to service which needs them more and then we can auto scale the whole server based upon load. We can have in-memory queues between those services. Do we have a software which can help in dynamically allocating more resources to a service if the no. of items in their queue is high?

  • Using a simple message broker to communicate between microservices is a good start but most of the time is not enough. If the computation in different services aren't totally independent then what would you do if ServiceN fails? How do you coordinate the ServiceN-1, ServiceN-2 ... rollback / apply compensation action / error handling flow? That's the area where orchestration (for example via Saga pattern) is needed. – Peter Csala Oct 20 '20 at 11:17
  • 1
    Depending on the your deployment setup the network latency can be negligible compared to the computation need. You need to measure to be sure. But you also have to prepare yourself to apply some sort of resilience strategy to overcome of the transient network failures. – Peter Csala Oct 20 '20 at 11:20
  • @PeterCsala Yes, that would be needed but if I use some sort of "Quality of service" within the server to distribute resources, I won't have to think of those concerns. – Deepak Mishra Oct 20 '20 at 11:39
2

I have a monolithic application which can be divided into multiple steps and different steps have variable scaling requirement

This has nothing to do with a state machine. This is a pipeline.

You probably should break it into microservices so you can scale per logical component. Why do you need a state machine? Do calls affect subsequent calls? Generally back-end calls are designed to NOT change state of other backend calls. If there is only a little state per call, then just pass it along the microservice pipeline as parameters.

If you really need a truly global state, a microservice is probably the wrong answer.

EDIT in response to comments

In the comments, you say step 2 acts on step 1 (and step 3 acts on step 2). This is a pipeline. If package A goes into step 1 -> Spits out Package A1, package A1 is fed into step 2, becomes package A2. If this is the flow, then I don't see why you need a state machine. You need to dynamically scale "step containers".

The only concern which I feel is that if Service1 container instance and service2 container instances are in different servers/hosts, then there would be some network latency which would be multiplied by the no. of requests.

If you're using Google Cloud all the physical machines will likely be in one very well connected data center. In my experience, latency inside the data center has never been an issue with either Google Cloud or AWS.

So based upon my understanding the microservices architecture is not a good candidate for state machines, if we are looking for real time performance.

I don't understand the fixation on state machines. Why do you need one. In a cloud environment, a state machine is a way of sharing global state between instances. The way the question reads now I don't think you need one.

resources which can be used by those services i.e. allocating more resources to service which needs them more and then we can auto scale the whole server based upon load.

So you're essentially redoing Google Cloud's work for them. Google Cloud offers something called a pipeline, which sounds like exactly what you need. The point of both Google Cloud and AWS is to move away from "I bought the box" to "I bought computing power". Google Cloud will auto-scale the pipeline, bringing more instances of each microservice online as it's needed, and deleting it when load tapers off. Don't redo this - you're probably not as good at this as Google or Amazon. Let them handle it - that's what you're paying for!

  • Step 2 acts on the result of step 1 and step n on the result of step n-1. If service 1 and service2 are in different servers then there is a concern of network latency. – Deepak Mishra Oct 20 '20 at 14:27
  • I just called it state machine, you can call it pipeline, it doesn't matter to me. btw, do you mean data flow/etl pipeline? – Deepak Mishra Oct 20 '20 at 15:52
  • @DeepakMishra - a state machine is a specific thing. I linked to the Google Cloud product I think you need. FYI - both AWS and Google Cloud provide a state machine service. – sevensevens Oct 20 '20 at 17:09
  • Changed the state machine to pipeline – Deepak Mishra Oct 21 '20 at 7:47
  • I can't use public cloud. My client won't agree to it. – Deepak Mishra Oct 22 '20 at 7:15
0

Separating them into individual services so that they can scale independently - good idea. For real-time responses, adding requests to the message queue will not help - you are right.

Network latency would be there if you are making a call to another service regardless of the queue, you have to ensure the services are in close proximity, have a circuit breaker while making downstream calls to ensure responsiveness is not taking a hit.

keep those state services in the same server and maybe control the amount of resources which can be used by those services

This is precisely the reason to move to Microservices and scale horizontally - managing resources will get extremely hard on the single machine. Scaling vertically has limitations, in terms of cost, in terms of ease of scaling too. If your scale is not much, and don't see it increasing by much - you can consider this though, it's a lot simple but beware of its limitations.

Just a thought - You can think of deploying your services as a side-car. This will help reduce latency between services, provide different resource configurations, scale horizontally easily as well.

  • I am scaling the application horizontally only. At server level, I am just trying to give some different percentage of resources to the different services using some "Quality of service" mechanism, so that on an average the server can handle more no of requests. – Deepak Mishra Oct 20 '20 at 12:07
  • Unless you have different pricing tiers in the application where you have a different QoS definition, a basic load balancer that distributes load evenly should do. What you can do further is have a feedback mechanism based on which you can bring up more instances, for ex: if all your machines have a load of say 75%, you can bring up a couple more. That would be a lot simpler and almost any LB can be used. If you divide your nodes for some to have more resources to provide better QoS, tag your resource and have a weighted priority to those nodes. This is provided by most cloud LBs – skott Oct 20 '20 at 16:05
0

As always: It depends on what your load/task is.

The trap I'd see in your situation is, if you have a lot of small step working over a small amount of data, you may end up with the cost of transfering thing from one place to another being too much. (I remember reading an article about super computer, in which engineer said it was often cheaper to run a calculation localy than import a cached result from another super computer)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.