Questions tagged [aws]

AWS refers to Amazon Web Services.

Filter by
Sorted by
Tagged with
1 vote
0 answers
135 views

Combine/Sync Amazon S3 and MongoDB Atlas with Elastic Search

I have many continuously growing (through scrapping) collections in MongoDB Atlas. The documents in each collection follows the following schema: { "source_url": "<some url on the web>", "html":...
user avatar
1 vote
2 answers
199 views

Fire aws lambda at a specific time

Let's say I have a booking system where I want to fire an aws lambda function to send an email or message to users, 10 minutes before their booking starts. I have looked online and found few solutions....
user avatar
  • 121
4 votes
1 answer
193 views

Robust way to aggregate results from 2 AWS lambdas for a SPA

I'm new to AWS serverless applications and am looking for something like ocelot request aggregation on the AWS serverless stack. Assume I have two AWS lambdas that return data needed by a SPA: A and ...
user avatar
  • 149
2 votes
1 answer
196 views

Does putting a serverless application in a VPC make it more secure?

I have an application which uses lambda and fargate (task) containers for compute, EventBridge for communication, S3 and DynamoDB for storage and exposes data out via an API Gateway. From my ...
user avatar
0 votes
1 answer
73 views

Refactoring and simplify this infrastructure building code [closed]

I have a piece of code (a Facade) that creates every single infrastructure component on aws. private void synthesizeStack() { Deployer<Vpc> vpcDeployer = new VpcDeployer(this); Deployer&...
user avatar
1 vote
2 answers
509 views

Why should I use containers instead of deploying build artifacts directly to Azure App Services or AWS Elastic Beanstalk?

Recently, I have been told by others to look into containerization of my stateless web applications (in this case .NET Core 2.x and 3.1). All of my dependencies are retrieved from public and private ...
user avatar
  • 226
2 votes
2 answers
331 views

Allow users upload CSV files, and embed the data for a simple API

Looking for the correct architecture, that should take into consideration - future edge-cases, bugs and pitfalls, performance, cloud pricing, ease of building and maintaining and security. I have a ...
user avatar
2 votes
0 answers
378 views

Running a high availability PostgreSQL cluster on native AWS services only

Backstory: I am unable to use RDS, as I need to install cartridges in my PostgreSQL instances. I have been trying to pin down an architecture for PostgreSQL running on EC2 instances for a few days. ...
user avatar
  • 29
-1 votes
1 answer
59 views

IP Whitelisting AWS consumers

If consumers of my API have their software on AWS, their ip address is subject to change if they are scaling their services horizontally (adding more machines). This means that I can't whitelist a ...
user avatar
  • 165
1 vote
0 answers
388 views

AWS S3 Presigned URL limited to one object upload per URL - How to upload multiple images

I have a AWS Lambda function that a user makes a GET request to and it returns presigned URL. The user then uploads an image by making a PUT request to that URL. Since AWS S3 PUT requests limits ...
user avatar
-4 votes
1 answer
120 views

Use React to parse a file (serverless)

I have a design question and am hoping for some validation. I want to create a 'serverless' web app which will parse a csv file and return an XML file. I will have a single page react app hosted in ...
user avatar
2 votes
3 answers
592 views

Securing API for third party use

We have a set of microservices and would like to expose endpoints from a subset of these for third parties to use. To this end, we will build an API Gateway that acts as the access control mechanism ...
user avatar
  • 165
1 vote
1 answer
485 views

Scheduled AWS Lambda to Update Database via API Call

I'm planning on the following for a use case scenario for AWS Lambda and want to make sure I'm headed in the right direction and that there's not some better/easier solution out there. The database ...
user avatar
-4 votes
2 answers
476 views

Confused about AWS dev: How to use React with Python in my lambda functions?

I'm very new to the serverless and backend world. I've been looking at tutorials on youtube and on serverless-stack.com (which is really good btw) and they mostly seem to cover React frontend ...
user avatar
0 votes
0 answers
341 views

Implement multiple file upload with Dropzone.js , Lambda and S3

This is more a general question about this project I have on. We need to implement some UI for our costumers to upload multiple files (2000+) every month, so we can send them by email to another ...
user avatar
1 vote
0 answers
41 views

Managing config (environment variables) shared by many applications (lambda functions)

In AWS I've got lots (dozens) of lambda functions. As my organization has gained experience with AWS, we've gone through various generations of infrastrucure around building and maintaining our ...
user avatar
0 votes
1 answer
3k views

What is the use of ENI in AWS?

This question is more theoretical. I am studying to certify in AWS, I am reading for the second time the certified solutions architect study guide of aws and it is not clear to me what is the use that ...
user avatar
  • 121
0 votes
1 answer
125 views

Is AWS Elasticsearch is completely managed service or is it just a Elasticsearch installation on set of ec2 instance

I want to understand the difference between AWS Elasticsearch service vs on-premise version. Does AWS provides a fully managed service for Elasticsearch (like DynamoDB, SQS etc) Do we have to manage ...
user avatar
2 votes
2 answers
2k views

Retry a failed REST API request - Java/Postgres

We have a REST API which calls a third-party REST API to Send Emails. The Third Party API is not super reliable and randomly fails every now and then with a 500. Our Clients do not want to retry at ...
user avatar
  • 139
1 vote
2 answers
155 views

How should I design an architecture for an endpoint to take in huge amount of data without loading the database?

Every day, there will be a set of new data that I will need to write and update to the database. This set of data can contain 8-10million records. This new set of data comes from Service A and it ...
user avatar
  • 887
4 votes
3 answers
645 views

how to pass credentials to api gateway securely?

I have a code at my local machine, with which I want to call an AWS Lambda function. I have configured AWS API Gateway as doorway to Lambda function, but I am concerned about security when sending ...
user avatar
3 votes
1 answer
618 views

AWS Multitenancy Solution with RDS?

I am looking for the best way of implementing a Multitenant App in AWS. Imagine I am an eCommerce business that has several eCommerce clients and those clients have other clients that are the ones ...
user avatar
0 votes
2 answers
813 views

Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?

We have a use case where an AWS Lambda function is called from AWS API Gateway on user request from the browser, it fetches data from the database and returns back to the client. The amount of time ...
user avatar
  • 3
0 votes
1 answer
1k views

Testing a message queue worker process?

Say you have an AWS SQS queue. Your Publisher class and Receiver class is already unit tested. Now you have a worker process that uses the Receiver class to actually receive the messages from SQS. My ...
user avatar
  • 313
2 votes
0 answers
26 views

Deployment pipeline as a component of an application

I am writing a serverless application using AWS SAM. I am wondering, would it be a good idea to include in my template a definition of a CodePipeline (and CodeBuild for that matter) for deploying that ...
user avatar
  • 121
4 votes
1 answer
689 views

How should AWS Lambda functions subscribe to domain events?

I'm doing my first Domain Driven Design (DDD) implementation. My architecture is comprised of .Net Core for defining my domain model, services, and building AWS Lambda functions. I am using Lambda ...
user avatar
1 vote
2 answers
262 views

Downloading and appending data from AWS S3 whilst avoiding clashes

I have to upload a TSV file to an S3 bucket, the content is form submission data. Another service will consume and do things with this data as soon as possible. The service would like one large file ...
user avatar
2 votes
1 answer
71 views

Ideal architecture for ML training -> API service workflow, with multiple models/services? [closed]

I'm planning to construct a workflow/environment for training and serving NLP classifiers, that follows something like: The model training system takes in annotated documents from a subset of a ...
user avatar
0 votes
0 answers
47 views

State Management design

I am working a product which have WebApi Android/ios app Web app WebApi is hosted in AWS EC2 . Currently we have requirement like we need to mantain state. Take one scenario. App have 4 pages. ...
user avatar
  • 99
4 votes
1 answer
579 views

How to organize projects in AWS?

In our team, we are using AWS as our main cloud provider and currently, we have 3 projects hosted on their platform. We are about to have 2 more projects in the next weeks, but first, we want to ...
user avatar
  • 201
-2 votes
1 answer
73 views

What is the right design for UI apps backed by Cloud services like AWS

I am looking for some general architectural guidance about designing good cloud apps, primarily using AWS. For the sake of argument, lets say it's a single page app, written in JS, hitting a couple ...
user avatar
  • 151
0 votes
1 answer
414 views

How would one log into an AWS User Pool from Unreal Engine?

I'm very, very new to the networking and AWS side of things, but experimenting a bit with some concepts. I was wondering if anyone had suggestions for the following use case. I'm in an experimental ...
user avatar
2 votes
0 answers
42 views

AWS codepipeline deployment

I currently have an AWS Codepipeline that is setup to automatically pull an Express, NodeJS API and deploy it with Lambda and API Gateway. It pulls the code from my GitHub repo production branch and ...
user avatar
1 vote
1 answer
56 views

How to be more productive with CloudFormation? [closed]

When using Terraform instead of Cloudformation on AWS, I usually get immediate feedback on the terminal, which makes the work very productive. With CloudFormation, however, I tend to edit the file in ...
user avatar
3 votes
1 answer
772 views

DynamoDB - Event Store on AWS

I'm designing an Event Store on AWS and I chose DynamoDB because it seemed the best option. My design seems to be quite good, but I'm facing some issues that I can't solve. The Design Events Events ...
user avatar
2 votes
1 answer
2k views

What is the difference between AWS Kinesis and AWS Logstash?

I've been having the hardest time understanding the difference between AWS Kinesis and Logstash. From the description of Logstash it says Logstash is a light-weight, open-source, server-side data ...
user avatar
  • 123
0 votes
0 answers
64 views

How to restrict API endpoints which are deployed in AWS Application Load Balancer?

I created one webapi application which includes few modules like account, user etc and overall includes 10 or more endpoints. I deployed this in EC2 instances and configured in application load ...
user avatar
  • 99
-2 votes
1 answer
203 views

Can a Database Scale Up? [closed]

Microservices are the latest software development craze. All the kids are doing it. When I started hearing about the concept, the first advantage that came to mind was the single responsibility ...
user avatar
1 vote
1 answer
1k views

.NET Core Microservice messaging with Amazon SNS and SQS

So I've done some searching but can't seem to find a whole lot of suggestions on this topic. My question is what are some opinions on the best way to receive messages from an Amazon SQS queue on a ....
user avatar
  • 158
7 votes
1 answer
3k views

Invoking lambda directly from S3 vs going via SQS

I'm relatively new to AWS. I'm working with what I think is a common pattern: Put file in S3 bucket Do something with said file in Lambda function I see two options for making this link (ignoring ...
user avatar
3 votes
1 answer
1k views

AWS SQS FIFO Limit Workaround

According to the AWS feature page for SQS: FIFO queues support up to 300 messages per second Standard queues support a nearly unlimited number of transactions per second (TPS) per API action. I'm ...
user avatar
0 votes
1 answer
63 views

The right way of doing images manipulation in a serverless app

This question is not specific to images, but this is the current issue im having so im using this as an example. So basically, I have some kind of a pipeline that some data(in this case, an image), ...
user avatar
  • 203
4 votes
1 answer
644 views

Sharding and application routing cross region on AWS

We current have a very simple Multi Tenant monolith, with a SQL SERVER backend (Self hosted on EC2 on AWS), and multiple application services talking to one DB behind an Classic AWS ELB. Our database ...
user avatar
5 votes
0 answers
2k views

Getting AWS SNS Messages into Azure Service Bus

I have a similar question to one already asked about SQS to Azure Service Bus - https://stackoverflow.com/questions/51650073/consume-aws-sqs-and-publish-to-azure-service-bus. I want to get messages ...
user avatar
-3 votes
1 answer
158 views

How to start designing a complex information system?

Recently I've been assigned to a new project to start with designing and visualizing business processes. Unfortunately I just can't wrap my head around this one, I understand what the client wants but ...
user avatar
1 vote
1 answer
152 views

What's stopping Cloud Platforms from owning the whole deployment process?

Although things have come a long way in recent times, in my workplace we're still talking about DLLs, containers, configs, substitutions, pipelines - all complexities when it comes to shipping code. ...
user avatar
  • 304
0 votes
1 answer
510 views

How to route/implement port forwarding with AWS/Apache web server (httpd) with a Node.JS app on port 3000?

I'm trying to understand this and having a hard time. I usually use Ubuntu server, with Apache2, and this time I am not. So I'm a bit out of my element. I'm also a software developer and I don't do ...
user avatar
  • 329
1 vote
1 answer
2k views

Serverless event-sourced architecture using AWS offerings

This is my first post on the Software Engineering Stack Exchange so let me know if something is wrong with it. I'm looking into the serverless offerings of Amazon to try to figure out if that is the ...
user avatar
  • 13
0 votes
2 answers
153 views

How to build reports for an ecommerce webapp without slowing it down

We have a medium sized ecommerce website which is entirely custom built in using Java and MySQL database on AWS infrastructure. Over a period of past several years, our orders volume has grown ...
user avatar
  • 249
1 vote
1 answer
1k views

FTP upload on AWS Elastic Beanstalk

We have our REST API running on AWS Elastic Beanstalk (NodeJS/Linux), there are some legacy devices that their only way to communicate with the world is uploading a file using FTP. These devices ...
user avatar