I am building a system composed of a few micro-services over AWS. I encountered the need of a certain MS to do the same logical work but of a big range of work load. for example, the same logical work could need 2vcpu and 4GB ram and another will need 4vcpu and 16GB ram. What is the best practice here? have all the instances big enough to fit my needs? have 2 ECS services one of 2vcpu and one 4vcpu having different SQS deliver the work load? or is there other rational idea here?
Usually, the assumption for a microservice architecture is that we want to be able to scale horizontally. This only works well if there are no differences between instances of the microservice. In particular, sessions or workloads should not be pinned to a particular instance, and it should be irrelevant which instance handles a task.
However, that is not essential.
What is essential for a service-oriented architecture is that you offer some network-based API to users of your service (e.g. a REST API). How your service then handles tasks internally is an implementation detail. It is absolutely OK if your service internally uses different sub-services, as long as the user of your service doesn't have to deal with this. For example, your service could use a gateway that forwards tasks to appropriate servers.
Is this a good idea? Hard to tell. It depends so much on circumstances.
If simplicity of the overall system is more important, keep all instances of the service the same. Make sure that each instance is able to handle every task. Keeping the infrastructure simple is especially important when this service isn't your primary responsibility. There are still some opportunities for cost optimization here, e.g. shutting down unneeded cloud-based instances depending on the load.
Using a bunch of chonky servers can make sense even when a fleet of tiny servers would also work. If a large instance can either handle one big task or two small tasks at a time, then just running two large instances will be easier and far more flexible than a 1×large + 2×small configuration – possibly at the same price.
If you are able to focus only on this one service and if small improvements multiply to large cost savings across your entire organization, then using a more complex internal architecture may be worth it. This could include heterogeneous infrastructure. But you only have a good business case for such cleverness if the extra development costs are amortized within a reasonable time frame. Quite often, it will be cheaper to just throw more hardware at the problem.