I've created a small prototype browser plugin and am now thinking about making it accessible to the public. This brings up an important question about gatekeeping API access and the right way to deploy a potentially expensive service.

The plugin uses Firebase for user authentication and a database. It also needs access to a custom machine learning model. I've set up this model in a Docker container which runs a FastAPI server on localhost. It's a transformer neural network, plus some python code for extra processing. The python code is trivial and could easily be reimplemented in another language. The network needs a lot of compute and each request should be considered expensive.
So far, I've set up authentication with Firebase, to restrict access to the database. But the FastAPI routes (which are stateless and don't reveal any user data) are not authenticated. When making this service public, I need some form of gatekeeping which allows me to limit the number of requests, prevent unauthenticated users, etc.

I'm not sure what the best practices for deployment are. Here's what I've been considering:

  • Find a provider that hosts the Docker container. Set up user authentication with a FastAPI route, which provides a JWT token. Only allow access to the expensive machine learning API with a valid token. The drawbacks: I think then I'd also have to move the database into the docker container. I don't want to force my users to log in twice, into separate services. And sharing the login credentials or the JWT token across two services doesn't seem feasible. My quick&dirty solution for this would be to hook up sqlite to FastAPI. But 1) I dont' want to worry about data loss. Firebase has convenient features for periodic backup etc. Sure, I can set this up myself, too. But I think proper database management quickly becomes a rabbit hole. And 2) the database traffic will be much much higher than the machine learning API traffic. I suspect that this will increase hosting costs: I'd need a very big virtual machine with a lot of bandwith. If I keep the two components separate, then I need a high-bandwith BAAS database plus a low-bandwith but high-compute ML API.

  • Set up a two-hop architecture. The machine learning API is not accessible to the client, but only to the cloud functions (or edge functions) of my BAAS provider. There was a good article from supabase https://supabase.com/docs/guides/ai/hugging-face. The supabase article talks specifically about huggingface, but this approach seems universal. Basically: Find a hosting provider for the Docker container and create a secret key. This key is kept in the codebase for the edge functions of the BAAS. An authenticated client can invoke such a function, but it doesn't see the code of the function itself, and therefore can't obtain the key. This means the expensive API calls are always routed through my own code, and I can rate limit or otherwise restrict them as I see fit. The problem here is: I find it hard to compute the pricing. And I worry that my whole software architecture will start to rely on a very large number of edge function invocations. This would make me vulnerable to sudden pricing changes from the provider.

This seems like a rather basic problem, and I've searched the site for related questions. So far I didn't find anything for this specific use-case of a high-bandwith API plus a high-compute API, and potentially using edge functions.

1 Answer 1


I recommend going with "Set up a two-hop architecture" option you provided. This approach adds an extra layer of security and control, as the costly machine learning API is not directly exposed to the client. You can implement rate-limiting, logging, or any other security measures at the cloud function level.

This architecture allows you to keep your database and machine learning API separate, enabling you to scale each component independently based on its own requirements. It also mitigates the risk of having to manage two separate authentication systems, as you mentioned with the "JWT and Fast API" option.

To address your concerns about pricing and provider lock-in, you can design your architecture in a way that allows you to easily switch providers. Use environment variables for configuration and keep provider-specific code isolated. Most cloud providers offer cost calculators to help you estimate expenses, and you can set up monitoring and alerts to keep track of usage and any unauthorized access attempts.

I think your approach offers a more secure and scalable solution that aligns well with best practices for API gatekeeping.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.