I work for an organization that heavily leverages AWS. There is a strong push that every team move from containers deployed on ECS to leverage AWS Lambda and step functions for (almost) every project. I know that there are workflows for which lambdas are the best solution, for example if you are running infrequent, short duration processes or processing S3 uploads for example. However I feel like my project isn't a great use case for them because:
We have many calls to a database and I don't want to have to worry about having to re-establish connections because the container a lambda was running in isn't available anymore.
We have many independent flows which would require too many lambdas to manage efficiently. With each new lambda you create you have to maintain an independent deployment pipeline and all the bureaucratic processes and items that go with owning a deploy-able component. By limiting the number of these the team can focus on delivering value vs maintenance.
We run a service that needs to be available 24/7 with Transactions Per Second around 10 to 30 around the clock. The runtime for each invocation is generally under 10 seconds with total transactions for a day in the 10's of thousands.
Also generally, I'm not bought into the serverless ecosystem because of a few pain points:
Local development. I know the tooling for developing AWS Lambdas on a developer machine has gotten much better, but having to start all these different lambdas locally with a step function to test an application locally seems like a huge hassle. I think it makes much more sense to have a single Java Spring Boot application with a click of a button you can test end to end and debug if necessary.
Reduced Isolation. If you have two ECS clusters and one is experiencing a huge throughput spike, the other ECS cluster will not be impacted because they are independent. Not so for lambda. We've seen that if other lambdas are using all the excess provisioned concurrency and we have to go over our reserved concurrency limit, then we are out of luck and we'll be rate limited heavily leading to errors. I know this should be a niche scenario, but why risk this at all? I think the fact that lambdas are not independent is one of things I like least about this ecosystem.
Am I thinking about lambdas/ serverless wrong? I am surrounded by developers who think that Java and Spring are dead and virtually every project must be built as a go/python lambda going forward.
@Mods if there are any ways that I can make this question more appropriate for the software engineering stack exchange community or rephrase it, I'm happy to make changes here as well.
Here's some links to research I've done so far on the topic: