I am building a system composed of a few micro-services over AWS. I encountered the need of a certain MS to do the same logical work but of a big range of work load. for example, the same logical work could need 2vcpu and 4GB ram and another will need 4vcpu and 16GB ram. What is the best practice here? have all the instances big enough to fit my needs? have 2 ECS services one of 2vcpu and one 4vcpu having different SQS deliver the work load? or is there other rational idea here?

  • why are there so drastically different resource requirements for the same task? If it is so different is it really still the same task? How would the correct instance be selected? Would it be up to the user of your service, or would your service determine that internally?
    – amon
    May 4, 2020 at 6:50
  • Data processing, could be small amounts of data for a very small customer or a lot of data for very large customer. Different Qs for Different workloads. QNormal for regular work loads QXL for large work loads
    – Amorphis
    May 4, 2020 at 7:04

1 Answer 1


Usually, the assumption for a microservice architecture is that we want to be able to scale horizontally. This only works well if there are no differences between instances of the microservice. In particular, sessions or workloads should not be pinned to a particular instance, and it should be irrelevant which instance handles a task.

However, that is not essential.

What is essential for a service-oriented architecture is that you offer some network-based API to users of your service (e.g. a REST API). How your service then handles tasks internally is an implementation detail. It is absolutely OK if your service internally uses different sub-services, as long as the user of your service doesn't have to deal with this. For example, your service could use a gateway that forwards tasks to appropriate servers.

Is this a good idea? Hard to tell. It depends so much on circumstances.

  • If simplicity of the overall system is more important, keep all instances of the service the same. Make sure that each instance is able to handle every task. Keeping the infrastructure simple is especially important when this service isn't your primary responsibility. There are still some opportunities for cost optimization here, e.g. shutting down unneeded cloud-based instances depending on the load.

    Using a bunch of chonky servers can make sense even when a fleet of tiny servers would also work. If a large instance can either handle one big task or two small tasks at a time, then just running two large instances will be easier and far more flexible than a 1×large + 2×small configuration – possibly at the same price.

  • If you are able to focus only on this one service and if small improvements multiply to large cost savings across your entire organization, then using a more complex internal architecture may be worth it. This could include heterogeneous infrastructure. But you only have a good business case for such cleverness if the extra development costs are amortized within a reasonable time frame. Quite often, it will be cheaper to just throw more hardware at the problem.

  • thanks for the answer. All those down votes I bet they never had to reduce Production costs. you're second points seems more fitting, its obvious to me that I can't have all instances to fit the largest work load as the cost will be enormous and yes I can have a large instance load work on 2-3 jobs at a time if small and 1 if large but that introduces more complexity to the system as the instance will have to wait till free to run as 1 work unit.
    – Amorphis
    May 4, 2020 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.