Questions tagged [aws]
AWS refers to Amazon Web Services.
105
questions
2
votes
2answers
233 views
Allow users upload CSV files, and embed the data for a simple API
Looking for the correct architecture, that should take into consideration - future edge-cases, bugs and pitfalls, performance, cloud pricing, ease of building and maintaining and security.
I have a ...
2
votes
0answers
63 views
Running a high availability PostgreSQL cluster on native AWS services only
Backstory: I am unable to use RDS, as I need to install cartridges in my PostgreSQL instances.
I have been trying to pin down an architecture for PostgreSQL running on EC2 instances for a few days. ...
-1
votes
1answer
39 views
IP Whitelisting AWS consumers
If consumers of my API have their software on AWS, their ip address is subject to change if they are scaling their services horizontally (adding more machines).
This means that I can't whitelist a ...
0
votes
0answers
18 views
AWS S3 Presigned URL limited to one object upload per URL - How to upload multiple images
I have a AWS Lambda function that a user makes a GET request to and it returns presigned URL. The user then uploads an image by making a PUT request to that URL. Since AWS S3 PUT requests limits ...
-2
votes
0answers
27 views
JAVA client - AWS EC2 server - continuous computing
I am trying to build a client server application using Java Client and AWS server. Client has an interactive GUI via which the user can perform a large number of operations on a data which is uploaded ...
-2
votes
0answers
12 views
Make Lambda and Ec2 instance communicate internally
I got this Machine Learning project installed on Ec2 and now I wanted to create a API with Lambda for other people to make predictions.
1- How could I make this communication happen internally? For ...
-4
votes
1answer
79 views
Use React to parse a file (serverless)
I have a design question and am hoping for some validation. I want to create a 'serverless' web app which will parse a csv file and return an XML file.
I will have a single page react app hosted in ...
2
votes
3answers
128 views
Securing API for third party use
We have a set of microservices and would like to expose endpoints from a subset of these for third parties to use. To this end, we will build an API Gateway that acts as the access control mechanism ...
0
votes
0answers
30 views
Design choices for federated contribution and consumption of workflow steps
Currently we have 5 different workflow steps and the number of those steps are growing and they are being built by different teams. There are multiple use cases for us to combine those steps in ...
1
vote
1answer
32 views
Scheduled AWS Lambda to Update Database via API Call
I'm planning on the following for a use case scenario for AWS Lambda and want to make sure I'm headed in the right direction and that there's not some better/easier solution out there.
The database ...
-4
votes
2answers
33 views
Confused about AWS dev: How to use React with Python in my lambda functions?
I'm very new to the serverless and backend world. I've been looking at tutorials on youtube and on serverless-stack.com (which is really good btw) and they mostly seem to cover React frontend ...
0
votes
0answers
14 views
Using custom password to create RDS Aurora PG instance and use it to connect to PgAdmin
Background/Use-Case: I am currently working on a requirement that includes creation of an RDS Aurora PG instance, to which the data will
be written through a Lambda.
I am creating the RDS using a ...
0
votes
0answers
24 views
Implement multiple file upload with Dropzone.js , Lambda and S3
This is more a general question about this project I have on. We need to implement some UI for our costumers to upload multiple files (2000+) every month, so we can send them by email to another ...
1
vote
0answers
27 views
Managing config (environment variables) shared by many applications (lambda functions)
In AWS I've got lots (dozens) of lambda functions. As my organization has gained experience with AWS, we've gone through various generations of infrastrucure around building and maintaining our ...
0
votes
0answers
68 views
How do I cheaply run different-sized same-algorithm workloads in EKS?
Problem Statement
I am trying to build a game engine where:
The game state exists in Postgres (probably AWS RDS for PostressSQL)
The simulation / game state is updated asynchronously by back-end ...
-1
votes
1answer
62 views
What is the use of ENI in AWS?
This question is more theoretical. I am studying to certify in AWS, I am reading for the second time the certified solutions architect study guide of aws and it is not clear to me what is the use that ...
0
votes
1answer
63 views
Is AWS Elasticsearch is completely managed service or is it just a Elasticsearch installation on set of ec2 instance
I want to understand the difference between AWS Elasticsearch service vs on-premise version. Does AWS provides a fully managed service for Elasticsearch (like DynamoDB, SQS etc) Do we have to manage ...
1
vote
2answers
470 views
Retry a failed REST API request - Java/Postgres
We have a REST API which calls a third-party REST API to Send Emails. The Third Party API is not super reliable and randomly fails every now and then with a 500.
Our Clients do not want to retry at ...
1
vote
2answers
99 views
How should I design an architecture for an endpoint to take in huge amount of data without loading the database?
Every day, there will be a set of new data that I will need to write and update to the database. This set of data can contain 8-10million records.
This new set of data comes from Service A and it ...
4
votes
3answers
189 views
how to pass credentials to api gateway securely?
I have a code at my local machine, with which I want to call an AWS Lambda function.
I have configured AWS API Gateway as doorway to Lambda function, but I am concerned about security when sending ...
3
votes
1answer
87 views
AWS Multitenancy Solution with RDS?
I am looking for the best way of implementing a Multitenant App in AWS. Imagine I am an eCommerce business that has several eCommerce clients and those clients have other clients that are the ones ...
0
votes
0answers
36 views
Scaling Dashboard Metrics and Analytics Using DynamoDB Data
We have an application that uses DynamoDB as its transactional database (OLTP). This is our preferred solution as we need to handle 1000s of key-value reads/writes per second and like the schemaless ...
0
votes
0answers
22 views
AWS SAM model for calling DDD functionality
I'm working my way through learning and implementing DDD. My domain is a ROI Calculator. I have an event store in place saving all the events that can happen with ROI Calculator. An instance of ROI ...
0
votes
2answers
109 views
Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?
We have a use case where an AWS Lambda function is called from AWS API Gateway on user request from the browser, it fetches data from the database and returns back to the client.
The amount of time ...
0
votes
1answer
151 views
Testing a message queue worker process?
Say you have an AWS SQS queue. Your Publisher class and Receiver class is already unit tested.
Now you have a worker process that uses the Receiver class to actually receive the messages from SQS. My ...
2
votes
0answers
23 views
Deployment pipeline as a component of an application
I am writing a serverless application using AWS SAM. I am wondering, would it be a good idea to include in my template a definition of a CodePipeline (and CodeBuild for that matter) for deploying that ...
3
votes
1answer
125 views
How should AWS Lambda functions subscribe to domain events?
I'm doing my first Domain Driven Design (DDD) implementation. My architecture is comprised of .Net Core for defining my domain model, services, and building AWS Lambda functions. I am using Lambda ...
1
vote
2answers
129 views
Downloading and appending data from AWS S3 whilst avoiding clashes
I have to upload a TSV file to an S3 bucket, the content is form submission data. Another service will consume and do things with this data as soon as possible. The service would like one large file ...
1
vote
1answer
45 views
Ideal architecture for ML training -> API service workflow, with multiple models/services? [closed]
I'm planning to construct a workflow/environment for training and serving NLP classifiers, that follows something like:
The model training system takes in annotated documents from a subset of a ...
0
votes
0answers
29 views
State Management design
I am working a product which have
WebApi
Android/ios app
Web app
WebApi is hosted in AWS EC2 .
Currently we have requirement like we need to mantain state. Take one scenario. App have 4 pages. ...
2
votes
1answer
167 views
How to organize projects in AWS?
In our team, we are using AWS as our main cloud provider and currently, we have 3 projects hosted on their platform.
We are about to have 2 more projects in the next weeks, but first, we want to ...
-2
votes
1answer
69 views
What is the right design for UI apps backed by Cloud services like AWS
I am looking for some general architectural guidance about designing good cloud apps, primarily using AWS. For the sake of argument, lets say it's a single page app, written in JS, hitting a couple ...
0
votes
1answer
114 views
How would one log into an AWS User Pool from Unreal Engine?
I'm very, very new to the networking and AWS side of things, but experimenting a bit with some concepts. I was wondering if anyone had suggestions for the following use case. I'm in an experimental ...
0
votes
0answers
93 views
How can I connect to the kubernetes pods to be able to get a VNC stream?
Our service has an admin panel, from which we need to be able to turn on the VNC stream (for example, via NoVNC) from the selected pod. The administrative panel is outside the Kubernetes cluster. I'm ...
2
votes
0answers
28 views
AWS codepipeline deployment
I currently have an AWS Codepipeline that is setup to automatically pull an Express, NodeJS API and deploy it with Lambda and API Gateway. It pulls the code from my GitHub repo production branch and ...
1
vote
1answer
50 views
How to be more productive with CloudFormation? [closed]
When using Terraform instead of Cloudformation on AWS, I usually get immediate feedback on the terminal, which makes the work very productive.
With CloudFormation, however, I tend to edit the file in ...
3
votes
1answer
397 views
DynamoDB - Event Store on AWS
I'm designing an Event Store on AWS and I chose DynamoDB because it seemed the best option. My design seems to be quite good, but I'm facing some issues that I can't solve.
The Design
Events
Events ...
0
votes
1answer
247 views
What is the difference between AWS Kinesis and AWS Logstash?
I've been having the hardest time understanding the difference between AWS Kinesis and Logstash.
From the description of Logstash it says
Logstash is a light-weight, open-source, server-side data ...
0
votes
0answers
36 views
How to restrict API endpoints which are deployed in AWS Application Load Balancer?
I created one webapi application which includes few modules like account, user etc and overall includes 10 or more endpoints.
I deployed this in EC2 instances and configured in application load ...
-3
votes
1answer
116 views
Can a Database Scale Up? [closed]
Microservices are the latest software development craze. All the kids are doing it. When I started hearing about the concept, the first advantage that came to mind was the single responsibility ...
0
votes
0answers
68 views
AWS Step Functions for a synchron Lambda call from another Lambda
I'm trying to understand AWS Step Functions... so far I understand it as an orchestration among AWS Lambdas in case of distributed processing of a complex task: "Do this, then that, then wait, then ...
1
vote
0answers
259 views
Integration Testing on AWS
I have a AWS pipeline connecting multiple resources to do 1 task which looks like as follows:
The data is listened from a SNS topic and then moved to Lambda where it is processed. From lambda, step ...
1
vote
1answer
590 views
.NET Core Microservice messaging with Amazon SNS and SQS
So I've done some searching but can't seem to find a whole lot of suggestions on this topic. My question is what are some opinions on the best way to receive messages from an Amazon SQS queue on a ....
6
votes
1answer
1k views
Invoking lambda directly from S3 vs going via SQS
I'm relatively new to AWS. I'm working with what I think is a common pattern:
Put file in S3 bucket
Do something with said file in Lambda function
I see two options for making this link (ignoring ...
2
votes
1answer
735 views
AWS SQS FIFO Limit Workaround
According to the AWS feature page for SQS:
FIFO queues support up to 300 messages per second
Standard queues support a nearly unlimited number of transactions per second (TPS) per API action.
...
0
votes
1answer
57 views
The right way of doing images manipulation in a serverless app
This question is not specific to images, but this is the current issue im having so im using this as an example.
So basically, I have some kind of a pipeline that some data(in this case, an image), ...
3
votes
1answer
193 views
Sharding and application routing cross region on AWS
We current have a very simple Multi Tenant monolith, with a SQL SERVER backend (Self hosted on EC2 on AWS), and multiple application services talking to one DB behind an Classic AWS ELB. Our database ...
3
votes
0answers
550 views
Getting AWS SNS Messages into Azure Service Bus
I have a similar question to one already asked about SQS to Azure Service Bus - https://stackoverflow.com/questions/51650073/consume-aws-sqs-and-publish-to-azure-service-bus. I want to get messages ...
-2
votes
1answer
102 views
How to start designing a complex information system?
Recently I've been assigned to a new project to start with designing and visualizing business processes. Unfortunately I just can't wrap my head around this one, I understand what the client wants but ...
1
vote
1answer
140 views
What's stopping Cloud Platforms from owning the whole deployment process?
Although things have come a long way in recent times, in my workplace we're still talking about DLLs, containers, configs, substitutions, pipelines - all complexities when it comes to shipping code.
...
