Hello Fellow Programmers, We are converting our monolithic application into microservice based. One challenge which we are facing is that one of our component is stateful. We cannot make this component stateless as of now (Due to high migration cost) so we have decided to horizontally scale this particular component on the basis of tenant (Multi Tenant Architecture - We already have a notion of tenant in our application). For e.g. If we have 5 tenants and 2 server instances we can distribute the load by ensuring that tenant 1,2 and 3 are running on instance 1 whereas tenant 4 and 5 are running on instance 2 (You can assume the load is evenly distributed among tenants for now). Now we are facing some problems with the gateway which will be routing the requests to the appropriate instances.
- We need to make the gateway aware of tenant. Now a tenant to application instance mapping might change at runtime (due to a specific instance failure or for load redistribution). Which means we need to change the mappings of gateway at runtime. This can be done if we regenerate the gateway configuration file and reload the gateway but I haven't seen any library which makes this generation easy for any gateway which makes me think this might not be how a gateway should be used and we might land into some problems in future.
- This approach might have been easier (Not requiring any form of generation) If the gateways would have provided a feature to resolve a variable dynamically instead of static mapping. For e.g. instead of mapping the url to tenant, we would map a url to application instance and that instance can be resolved from tenant using a function call done by the gateway. But no gateway (Atleast the ones that I have explored) provide such feature. I have a workaround where I am using custom predicate in spring cloud gateway to dynamically resolve the mapping from local cache but I am not sure If this is how I am supposed to use a custom predicate feature of Spring gateway as I haven't seen any examples or papers where a custom predicate has been used in such a way.
- I searched the internet to find how people solve multi tenant based application level routing. I can't find any blogs or papers on this topic. All I see is the application being stateless and multi tenancy is being handled at database level. But unfortunately making the current application stateless would require lot of changes and we do not have bandwidth for that.
I would like to know If someone has solved this problem before or can guide me someplace where I can find the solution for this. There are chances that we might be doing something wrong fundamentally which is why I am struggling to find solutions on internet and I would like to know what they are. Thank you in advance, I would really appreciate If someone could write their opinion on this.