Questions tagged [aws]

AWS refers to Amazon Web Services.

Filter by
Sorted by
Tagged with
4 votes
4 answers
296 views

Do Lambdas(AWS) stop making sense after a certain number of functions?

I am working on a project and created multiple lambda functions and now I am thinking about whether I made the correct decision to go this route. Reasons I chose lambda: The App will have spiky ...
user avatar
  • 149
-3 votes
0 answers
26 views

Serverless architecture for an enterprise app [duplicate]

excited to be here. I'm trying to get some thoughts around serverless architecture that seems to be popular lately. I'm building a new SaaS solution and trying to decide whether serverless is ...
user avatar
  • 95
0 votes
0 answers
58 views

Best practices to deal with realtime data in REST API, Postgres and Redis

Overview Today we have a monolithic service that deals with realtime time data. The monolithic are composed by follow stack: Django + AuroraDB Cluster (Postgres AWS) + Varnish cache and cloudfront at ...
user avatar
1 vote
1 answer
102 views

Should I use different Buildspec files for branch builds and deployment builds?

Very recently a discussion came up regarding the usage of different buildspec.yaml files, one for branch builds and the other for deployment builds and I was wondering, since after some research I ...
user avatar
  • 13
0 votes
0 answers
44 views

Multitenet architecture with AWS cognito

I want to build a multi-tenant application with AWS Cognito. One user can belong to many tenants and they can have different roles in each tenant. Each tenant will have resources like S3 buckets and ...
user avatar
  • 101
1 vote
1 answer
149 views

Storing large data shared between microservices running in containers at AWS

I'm dealing with an application that does the following: we have a microservice running on EKS called by jobs that pull XML data off an external system. The data volume can be fairly large (700MB-1GB ...
user avatar
  • 946
0 votes
0 answers
35 views

Unity On-Demand Content Delivery. Addressable or Asset Bundles?

We have more than 1000 3D clothes (and they will probably grow) that we have to stream on our unity mobile application and right now we are trying to implement a content delivery solution for our app ...
user avatar
0 votes
1 answer
188 views

Removing successful cases from SQS batch. Is it wrong?

I have been working on a lambda that is triggered by a SQS with a batch (let's say of 5) and has automatic retry enabled. The most common problem with this kind of setup that I have seen is that, if ...
user avatar
0 votes
1 answer
187 views

Multiple Lambdas, one database?

Having read through various microservices literature, I've seen a common recommendation that a microservice should generally be fairly cohesive, owning its own data repository, and serving as the ...
user avatar
  • 111
0 votes
0 answers
48 views

Where to deploy a Flask Rest API with 1500 active users per day?

I will be working on a Flask Rest API and I don't know where to deploy my Flask app depending on number of the active users. This is my use case; My customer has an Oracle DB for their hospital and ...
user avatar
0 votes
0 answers
38 views

Do you put all CRUD Java Handler classes into a single AWS SAM template?

When building REST API with AWS Lambda + Java, do you define all lambda functions in a single AWS SAM template or create a separate project for each CRUD operation? Given that on AWS, each lambda ...
user avatar
0 votes
0 answers
193 views

AWS Java Lambda Class Naming Convention?

I wonder if there is a naming convention or a best practice to name Java Classes when building AWS Lambda functions with Java for REST API? For example, if I have a Java project that contains Lambda ...
user avatar
0 votes
1 answer
62 views

Table design question - DynamoDB the right choice?

Our team has started to use AWS and one of our projects will require storing approval statuses of various recommendations in a table. There are various things that identify a single recommendation, ...
user avatar
0 votes
1 answer
88 views

Is it a good practice to add a range key to an AWS dynamo table even if there is no need in current specification?

I am using AWS dynamodb for my project. One can choose to use a simple key (partition key alone) or a composite key (partition key + range key) when creating a dynamo db table. But once it is created, ...
user avatar
1 vote
0 answers
36 views

List Git branch currently on each EC2

We have a handful of AWS EC2s set up for testing ongoing feature developments. I'm wondering if anyone knows an easy, automated way to keep track of which git branch we're testing on each EC2. Maybe ...
user avatar
0 votes
1 answer
45 views

Approaches for getting variables from AWS Parameter Store into app

I'm converting an app to run on AWS Fargate and was going to use the Parameter Store to hold configuration values. Most of the information I've seen about it revolves around injecting Parameter Store ...
user avatar
  • 946
0 votes
1 answer
298 views

Refresh Token Storage on cloud best practice?

Background I am building a web app that allows the user to integrate with multiple services like Google, Twitter, Github etc. using OAuth2.0. Currently, I retrieve the refresh token on sign-in to ...
user avatar
  • 101
8 votes
3 answers
4k views

How to handle backpressure in message queue

I am designing a AWS web service which is going to get 1000 TPS from devices(Android) and it has dependency on multiple downstream services. The usecase is to hit this service periodically from device,...
user avatar
0 votes
0 answers
45 views

How should I handle keys in production

I'm creating an authorisation service, which does signing using a key. For local development I was randomly generating a key on application start-up. Now I'm deploying to the cloud (currently going ...
user avatar
1 vote
2 answers
160 views

In Publish-subscribe pattern can a microservice publish message for itself

I am designing a somewhat complex flow in publish subscribe pattern using AWS SNS, SQS , Microservice and Lambda functions. To break the whole flow into multiple logical & smaller sub-flows I am ...
user avatar
-1 votes
1 answer
137 views

Design pattern to sync local databases with the cloud [closed]

I would like to find the best way to solve the following problem: I need a database for a software. The software needs to use the data to populate the UI and allow the user to interact with them (edit,...
user avatar
  • 101
0 votes
0 answers
20 views

AWS Serverless Service to trigger an event when something doesn't happen

I'd like to use some sort of AWS serverless technology to trigger an event when a clickstream stops receiving clicks. Imagine a website visitor clicking around a site. Within our current solution ...
user avatar
0 votes
0 answers
35 views

Kubernetes auto scaling

We are currently in the process of doing an infrastructure overhaul. A bit of background of what our business model currently is: We are an aggregator of bills and payment services for businesses. ...
user avatar
1 vote
0 answers
39 views

Designing an architecture for running multiple scripts

Our company is an aggregator of multiple payment services fragmented on the web. So instead of businesses integrating multiple services one by one, they will only need to integrate to our API, and we ...
user avatar
0 votes
2 answers
156 views

How to build an event sourced system with AWS (Specifically, how to deal with side effects)

So the current setup is a typical event sourced system built on AWS: Lambdas exposed with API Gateway to get the orders (commands), the domain (aggregates) apply some validations after loading the ...
user avatar
  • 101
0 votes
1 answer
118 views

AWS CodeCommit - duplicate test and production repos

I am using AWS CodeCommit with CodePipeline. I have two totally separate AWS accounts, one for test and one for production, identical Code Pipelines setup in each account, each with a corresponding ...
user avatar
3 votes
2 answers
214 views

Foreign Keys and Eventual Consistency

We have a project that loads metric data from various etls and services into an AWS SQS queue, where that data is processed and inserted into a metrics database, also running on AWS RDS. When we ...
user avatar
  • 2,343
1 vote
2 answers
135 views

When creating a RESTful API in AWS, why would you use EC2 over a Lambda integration?

In AWS you can easily create a RESTful API using API Gateway. With a serverless approach, you would have API Gateway integrate with Lambda to handle your endpoint logic. This seems like the most ...
user avatar
  • 1,151
0 votes
1 answer
157 views

Large file uploading in chunks

How to upload large(mb/gb) of video files from client to app server? The solution as of now I know is pretty simple and widely used is to break file in chunks at at client side and send http post ...
user avatar
  • 81
0 votes
1 answer
170 views

AWS Architecture: How to search in Dynamodb?

Firstly, I want to state that this is my first time trying to build an app. I am doing this for fun, mostly as an exercise to learn system design/architecture. The goal is to build an app that would ...
user avatar
1 vote
0 answers
99 views

How to (simply) architecture a way to ingest multiple types of large files, process them, and send data in chunks to web services?

Note: All of this would be in AWS Hi everyone, What would you guys suggest for building something that: Takes in several different input file types (ex: csv, json, jsonl, xml, .gz, ...) That can be ...
user avatar
1 vote
1 answer
774 views

How to check if an AWS SageMaker endpoint already exists?

I am using the AWS SDK for Java V2, and I want to delete a SageMaker endpoint and a SageMaker endpoint config. What is the best way to do this? For example, I am currently first using the ...
user avatar
2 votes
0 answers
200 views

Determining when to use Serverless vs Containerized application (AWS Lambda vs ECS) - Is Java Spring dead?

I work for an organization that heavily leverages AWS. There is a strong push that every team move from containers deployed on ECS to leverage AWS Lambda and step functions for (almost) every project....
user avatar
-2 votes
2 answers
77 views

What's better, using DB or Storage services to save videos?

I am creating a website that allows to teachers to uploading courses, what's better, to just use the DB or should I use storage services, and how could I know the size of the videos? I am using AWS if ...
user avatar
9 votes
5 answers
507 views

Re-architecting CPU intensive Node application to handle multiple users

A few years ago, I wrote an application that allowed users to upload a file (it's a very specific file type) to a server. It then sends instructions to the server on how to visualise the data, and the ...
user avatar
  • 79
2 votes
1 answer
263 views

Should a Web Application File URL Have Public or Private Access

I am debating whether to give my files a public url or a limited private one. I am hosting various files for a mobile/web application. These will include product images and videos. Currently only ...
user avatar
  • 161
1 vote
2 answers
105 views

AWS Kubernetes - MySQL DB instance - RDS or Docker Mysql Image

We're trying to migrate to either a cheap hosting or aws Kubernetes to cut costs. Our website is a game site and has around 100 users. The servers include 2 mysql db, 2 tomcat server and a management ...
user avatar
  • 111
3 votes
1 answer
111 views

AWS Serverless Microservice design

I am building an open source task management and note taking service that runs on AWS. I'm building it mostly as a hobby in my spare time and will eventually ship a mobile app that consume it into an ...
user avatar
0 votes
1 answer
53 views

IOT data interpretation

This is a more architectural question and I am hoping to get more ideas out of this thread. I have a computer vision model detecting daily life activities and sending data to the server every second. ...
user avatar
-3 votes
2 answers
249 views

Cloudfront distribution having multiple domains

I am developing a multi-tenant project that will be a kind of micro "ecommerce" and therefore customers will be able to create their own stores and will be able to choose between creating a ...
user avatar
0 votes
1 answer
268 views

designing a secure/encrypted file storage for a multi-tenant saas app with local and cloud storage in mind

as the title suggest, I need to handle the tenant files in a multi-tenant saas app. fairly common scenario I believe and should have been solved by the gurus already. right now the app is in infancy ...
user avatar
  • 143
0 votes
0 answers
276 views

How to implement a debounce buffer system on AWS?

I'll describe my case: I have an aws lambda that will expose an API interface that will return json data (by topic) which is cached. This lambda will also offer private endpoints to recalculate the ...
user avatar
  • 493
-2 votes
1 answer
53 views

A Recipe for Handling an AWS Network Request from Start to Finish at Global Scale

I have been reading the AWS documentation for a solid 2 weeks, and configuring a terraform system to deploy a multi-region network. I have it mostly wired up, but I am not sure I have accounted for ...
user avatar
  • 2,412
2 votes
0 answers
26 views

In a multi-region network, how do you organize requests through VPNs for atomic database transaction requirements?

I am trying to setup a multi-region (global) AWS system, where each region (which has multiple availability zones) has its own VPC. In theory these VPCs all use different portions of the private IP ...
user avatar
  • 2,412
2 votes
1 answer
151 views

Serverless game server architecture

First post on the software engineering stack exchange, I hope the question fits the purpose of this sub. I'm building a serverless game server with the following architecture and was wondering if ...
user avatar
  • 129
0 votes
1 answer
97 views

Reading from SQS using AWS Lambda vs WebApp background worker

I'm writing a microservice (in .netcore) that will handle some requests using WebAPI, but also it needs to react to some external events. I see 2 possibilities how to solve it (on a very high level): ...
user avatar
0 votes
1 answer
119 views

Maintaining Objects Across API Deployment Instances

I am working on a web application as a hobby and trying to learn some concepts related to cloud development and distributed applications. I am currently targeting an AWS EC2 instance as a deployment ...
user avatar
  • 111
1 vote
1 answer
60 views

What is the threat model for deciding between unencrypted vs. encrypted EBS volumes?

Let me start by saying that I'm not questioning the utility of encrypting EBS volumes, nor asking how it works. I'm just wondering what specifically encrypting EBS volumes is protecting against? For ...
user avatar
-4 votes
1 answer
39 views

Microservices of different resource type

I am building a system composed of a few micro-services over AWS. I encountered the need of a certain MS to do the same logical work but of a big range of work load. for example, the same logical work ...
user avatar
1 vote
0 answers
135 views

Combine/Sync Amazon S3 and MongoDB Atlas with Elastic Search

I have many continuously growing (through scrapping) collections in MongoDB Atlas. The documents in each collection follows the following schema: { "source_url": "<some url on the web>", "html":...
user avatar