Questions tagged [aws]

AWS refers to Amazon Web Services.

Filter by
Sorted by
Tagged with
2
votes
2answers
233 views

Allow users upload CSV files, and embed the data for a simple API

Looking for the correct architecture, that should take into consideration - future edge-cases, bugs and pitfalls, performance, cloud pricing, ease of building and maintaining and security. I have a ...
2
votes
0answers
63 views

Running a high availability PostgreSQL cluster on native AWS services only

Backstory: I am unable to use RDS, as I need to install cartridges in my PostgreSQL instances. I have been trying to pin down an architecture for PostgreSQL running on EC2 instances for a few days. ...
-1
votes
1answer
39 views

IP Whitelisting AWS consumers

If consumers of my API have their software on AWS, their ip address is subject to change if they are scaling their services horizontally (adding more machines). This means that I can't whitelist a ...
0
votes
0answers
18 views

AWS S3 Presigned URL limited to one object upload per URL - How to upload multiple images

I have a AWS Lambda function that a user makes a GET request to and it returns presigned URL. The user then uploads an image by making a PUT request to that URL. Since AWS S3 PUT requests limits ...
-2
votes
0answers
27 views

JAVA client - AWS EC2 server - continuous computing

I am trying to build a client server application using Java Client and AWS server. Client has an interactive GUI via which the user can perform a large number of operations on a data which is uploaded ...
-2
votes
0answers
12 views

Make Lambda and Ec2 instance communicate internally

I got this Machine Learning project installed on Ec2 and now I wanted to create a API with Lambda for other people to make predictions. 1- How could I make this communication happen internally? For ...
-4
votes
1answer
79 views

Use React to parse a file (serverless)

I have a design question and am hoping for some validation. I want to create a 'serverless' web app which will parse a csv file and return an XML file. I will have a single page react app hosted in ...
2
votes
3answers
128 views

Securing API for third party use

We have a set of microservices and would like to expose endpoints from a subset of these for third parties to use. To this end, we will build an API Gateway that acts as the access control mechanism ...
0
votes
0answers
30 views

Design choices for federated contribution and consumption of workflow steps

Currently we have 5 different workflow steps and the number of those steps are growing and they are being built by different teams. There are multiple use cases for us to combine those steps in ...
1
vote
1answer
32 views

Scheduled AWS Lambda to Update Database via API Call

I'm planning on the following for a use case scenario for AWS Lambda and want to make sure I'm headed in the right direction and that there's not some better/easier solution out there. The database ...
-4
votes
2answers
33 views

Confused about AWS dev: How to use React with Python in my lambda functions?

I'm very new to the serverless and backend world. I've been looking at tutorials on youtube and on serverless-stack.com (which is really good btw) and they mostly seem to cover React frontend ...
0
votes
0answers
14 views

Using custom password to create RDS Aurora PG instance and use it to connect to PgAdmin

Background/Use-Case: I am currently working on a requirement that includes creation of an RDS Aurora PG instance, to which the data will be written through a Lambda. I am creating the RDS using a ...
0
votes
0answers
24 views

Implement multiple file upload with Dropzone.js , Lambda and S3

This is more a general question about this project I have on. We need to implement some UI for our costumers to upload multiple files (2000+) every month, so we can send them by email to another ...
1
vote
0answers
27 views

Managing config (environment variables) shared by many applications (lambda functions)

In AWS I've got lots (dozens) of lambda functions. As my organization has gained experience with AWS, we've gone through various generations of infrastrucure around building and maintaining our ...
0
votes
0answers
68 views

How do I cheaply run different-sized same-algorithm workloads in EKS?

Problem Statement I am trying to build a game engine where: The game state exists in Postgres (probably AWS RDS for PostressSQL) The simulation / game state is updated asynchronously by back-end ...
-1
votes
1answer
62 views

What is the use of ENI in AWS?

This question is more theoretical. I am studying to certify in AWS, I am reading for the second time the certified solutions architect study guide of aws and it is not clear to me what is the use that ...
0
votes
1answer
63 views

Is AWS Elasticsearch is completely managed service or is it just a Elasticsearch installation on set of ec2 instance

I want to understand the difference between AWS Elasticsearch service vs on-premise version. Does AWS provides a fully managed service for Elasticsearch (like DynamoDB, SQS etc) Do we have to manage ...
1
vote
2answers
470 views

Retry a failed REST API request - Java/Postgres

We have a REST API which calls a third-party REST API to Send Emails. The Third Party API is not super reliable and randomly fails every now and then with a 500. Our Clients do not want to retry at ...
1
vote
2answers
99 views

How should I design an architecture for an endpoint to take in huge amount of data without loading the database?

Every day, there will be a set of new data that I will need to write and update to the database. This set of data can contain 8-10million records. This new set of data comes from Service A and it ...
4
votes
3answers
189 views

how to pass credentials to api gateway securely?

I have a code at my local machine, with which I want to call an AWS Lambda function. I have configured AWS API Gateway as doorway to Lambda function, but I am concerned about security when sending ...
3
votes
1answer
87 views

AWS Multitenancy Solution with RDS?

I am looking for the best way of implementing a Multitenant App in AWS. Imagine I am an eCommerce business that has several eCommerce clients and those clients have other clients that are the ones ...
0
votes
0answers
36 views

Scaling Dashboard Metrics and Analytics Using DynamoDB Data

We have an application that uses DynamoDB as its transactional database (OLTP). This is our preferred solution as we need to handle 1000s of key-value reads/writes per second and like the schemaless ...
0
votes
0answers
22 views

AWS SAM model for calling DDD functionality

I'm working my way through learning and implementing DDD. My domain is a ROI Calculator. I have an event store in place saving all the events that can happen with ROI Calculator. An instance of ROI ...
0
votes
2answers
109 views

Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?

We have a use case where an AWS Lambda function is called from AWS API Gateway on user request from the browser, it fetches data from the database and returns back to the client. The amount of time ...
0
votes
1answer
151 views

Testing a message queue worker process?

Say you have an AWS SQS queue. Your Publisher class and Receiver class is already unit tested. Now you have a worker process that uses the Receiver class to actually receive the messages from SQS. My ...
2
votes
0answers
23 views

Deployment pipeline as a component of an application

I am writing a serverless application using AWS SAM. I am wondering, would it be a good idea to include in my template a definition of a CodePipeline (and CodeBuild for that matter) for deploying that ...
3
votes
1answer
125 views

How should AWS Lambda functions subscribe to domain events?

I'm doing my first Domain Driven Design (DDD) implementation. My architecture is comprised of .Net Core for defining my domain model, services, and building AWS Lambda functions. I am using Lambda ...
1
vote
2answers
129 views

Downloading and appending data from AWS S3 whilst avoiding clashes

I have to upload a TSV file to an S3 bucket, the content is form submission data. Another service will consume and do things with this data as soon as possible. The service would like one large file ...
1
vote
1answer
45 views

Ideal architecture for ML training -> API service workflow, with multiple models/services? [closed]

I'm planning to construct a workflow/environment for training and serving NLP classifiers, that follows something like: The model training system takes in annotated documents from a subset of a ...
0
votes
0answers
29 views

State Management design

I am working a product which have WebApi Android/ios app Web app WebApi is hosted in AWS EC2 . Currently we have requirement like we need to mantain state. Take one scenario. App have 4 pages. ...
2
votes
1answer
167 views

How to organize projects in AWS?

In our team, we are using AWS as our main cloud provider and currently, we have 3 projects hosted on their platform. We are about to have 2 more projects in the next weeks, but first, we want to ...
-2
votes
1answer
69 views

What is the right design for UI apps backed by Cloud services like AWS

I am looking for some general architectural guidance about designing good cloud apps, primarily using AWS. For the sake of argument, lets say it's a single page app, written in JS, hitting a couple ...
0
votes
1answer
114 views

How would one log into an AWS User Pool from Unreal Engine?

I'm very, very new to the networking and AWS side of things, but experimenting a bit with some concepts. I was wondering if anyone had suggestions for the following use case. I'm in an experimental ...
0
votes
0answers
93 views

How can I connect to the kubernetes pods to be able to get a VNC stream?

Our service has an admin panel, from which we need to be able to turn on the VNC stream (for example, via NoVNC) from the selected pod. The administrative panel is outside the Kubernetes cluster. I'm ...
2
votes
0answers
28 views

AWS codepipeline deployment

I currently have an AWS Codepipeline that is setup to automatically pull an Express, NodeJS API and deploy it with Lambda and API Gateway. It pulls the code from my GitHub repo production branch and ...
1
vote
1answer
50 views

How to be more productive with CloudFormation? [closed]

When using Terraform instead of Cloudformation on AWS, I usually get immediate feedback on the terminal, which makes the work very productive. With CloudFormation, however, I tend to edit the file in ...
3
votes
1answer
397 views

DynamoDB - Event Store on AWS

I'm designing an Event Store on AWS and I chose DynamoDB because it seemed the best option. My design seems to be quite good, but I'm facing some issues that I can't solve. The Design Events Events ...
0
votes
1answer
247 views

What is the difference between AWS Kinesis and AWS Logstash?

I've been having the hardest time understanding the difference between AWS Kinesis and Logstash. From the description of Logstash it says Logstash is a light-weight, open-source, server-side data ...
0
votes
0answers
36 views

How to restrict API endpoints which are deployed in AWS Application Load Balancer?

I created one webapi application which includes few modules like account, user etc and overall includes 10 or more endpoints. I deployed this in EC2 instances and configured in application load ...
-3
votes
1answer
116 views

Can a Database Scale Up? [closed]

Microservices are the latest software development craze. All the kids are doing it. When I started hearing about the concept, the first advantage that came to mind was the single responsibility ...
0
votes
0answers
68 views

AWS Step Functions for a synchron Lambda call from another Lambda

I'm trying to understand AWS Step Functions... so far I understand it as an orchestration among AWS Lambdas in case of distributed processing of a complex task: "Do this, then that, then wait, then ...
1
vote
0answers
259 views

Integration Testing on AWS

I have a AWS pipeline connecting multiple resources to do 1 task which looks like as follows: The data is listened from a SNS topic and then moved to Lambda where it is processed. From lambda, step ...
1
vote
1answer
590 views

.NET Core Microservice messaging with Amazon SNS and SQS

So I've done some searching but can't seem to find a whole lot of suggestions on this topic. My question is what are some opinions on the best way to receive messages from an Amazon SQS queue on a ....
6
votes
1answer
1k views

Invoking lambda directly from S3 vs going via SQS

I'm relatively new to AWS. I'm working with what I think is a common pattern: Put file in S3 bucket Do something with said file in Lambda function I see two options for making this link (ignoring ...
2
votes
1answer
735 views

AWS SQS FIFO Limit Workaround

According to the AWS feature page for SQS: FIFO queues support up to 300 messages per second Standard queues support a nearly unlimited number of transactions per second (TPS) per API action. ...
0
votes
1answer
57 views

The right way of doing images manipulation in a serverless app

This question is not specific to images, but this is the current issue im having so im using this as an example. So basically, I have some kind of a pipeline that some data(in this case, an image), ...
3
votes
1answer
193 views

Sharding and application routing cross region on AWS

We current have a very simple Multi Tenant monolith, with a SQL SERVER backend (Self hosted on EC2 on AWS), and multiple application services talking to one DB behind an Classic AWS ELB. Our database ...
3
votes
0answers
550 views

Getting AWS SNS Messages into Azure Service Bus

I have a similar question to one already asked about SQS to Azure Service Bus - https://stackoverflow.com/questions/51650073/consume-aws-sqs-and-publish-to-azure-service-bus. I want to get messages ...
-2
votes
1answer
102 views

How to start designing a complex information system?

Recently I've been assigned to a new project to start with designing and visualizing business processes. Unfortunately I just can't wrap my head around this one, I understand what the client wants but ...
1
vote
1answer
140 views

What's stopping Cloud Platforms from owning the whole deployment process?

Although things have come a long way in recent times, in my workplace we're still talking about DLLs, containers, configs, substitutions, pipelines - all complexities when it comes to shipping code. ...