Questions tagged [aws]
AWS refers to Amazon Web Services.
0
votes
0answers
25 views
State Management design
I am working a product which have
WebApi
Android/ios app
Web app
WebApi is hosted in AWS EC2 .
Currently we have requirement like we need to mantain state. Take one scenario. App have 4 pages. ...
2
votes
1answer
76 views
How to organize projects in AWS?
In our team, we are using AWS as our main cloud provider and currently, we have 3 projects hosted on their platform.
We are about to have 2 more projects in the next weeks, but first, we want to ...
-2
votes
1answer
57 views
What is the right design for UI apps backed by Cloud services like AWS
I am looking for some general architectural guidance about designing good cloud apps, primarily using AWS. For the sake of argument, lets say it's a single page app, written in JS, hitting a couple ...
0
votes
1answer
38 views
How would one log into an AWS User Pool from Unreal Engine?
I'm very, very new to the networking and AWS side of things, but experimenting a bit with some concepts. I was wondering if anyone had suggestions for the following use case. I'm in an experimental ...
0
votes
0answers
14 views
How can I connect to the kubernetes pods to be able to get a VNC stream?
Our service has an admin panel, from which we need to be able to turn on the VNC stream (for example, via NoVNC) from the selected pod. The administrative panel is outside the Kubernetes cluster. I'm ...
0
votes
0answers
19 views
AWS codepipeline deployment
I currently have an AWS Codepipeline that is setup to automatically pull an Express, NodeJS API and deploy it with Lambda and API Gateway. It pulls the code from my GitHub repo production branch and ...
0
votes
0answers
25 views
How to be more productive with CloudFormation?
When using Terraform instead of Cloudformation on AWS, I usually get immediate feedback on the terminal, which makes the work very productive.
With CloudFormation, however, I tend to edit the file in ...
1
vote
1answer
86 views
DynamoDB - Event Store on AWS
I'm designing an Event Store on AWS and I chose DynamoDB because it seemed the best option. My design seems to be quite good, but I'm facing some issues that I can't solve.
The Design
Events
Events ...
0
votes
1answer
39 views
What is the difference between AWS Kinesis and AWS Logstash?
I've been having the hardest time understanding the difference between AWS Kinesis and Logstash.
From the description of Logstash it says
Logstash is a light-weight, open-source, server-side data ...
0
votes
0answers
28 views
How to restrict API endpoints which are deployed in AWS Application Load Balancer?
I created one webapi application which includes few modules like account, user etc and overall includes 10 or more endpoints.
I deployed this in EC2 instances and configured in application load ...
-3
votes
1answer
105 views
Can a Database Scale Up? [closed]
Microservices are the latest software development craze. All the kids are doing it. When I started hearing about the concept, the first advantage that came to mind was the single responsibility ...
0
votes
0answers
58 views
AWS Step Functions for a synchron Lambda call from another Lambda
I'm trying to understand AWS Step Functions... so far I understand it as an orchestration among AWS Lambdas in case of distributed processing of a complex task: "Do this, then that, then wait, then ...
1
vote
0answers
48 views
Integration Testing on AWS
I have a AWS pipeline connecting multiple resources to do 1 task which looks like as follows:
The data is listened from a SNS topic and then moved to Lambda where it is processed. From lambda, step ...
1
vote
1answer
270 views
.NET Core Microservice messaging with Amazon SNS and SQS
So I've done some searching but can't seem to find a whole lot of suggestions on this topic. My question is what are some opinions on the best way to receive messages from an Amazon SQS queue on a ....
0
votes
1answer
286 views
Invoking lambda directly from S3 vs going via SQS
I'm relatively new to AWS. I'm working with what I think is a common pattern:
Put file in S3 bucket
Do something with said file in Lambda function
I see two options for making this link (ignoring ...
0
votes
1answer
63 views
Architecture patterns or SaaS solutions for throttling outgoing messages? [on hold]
I'm implementing my service on AWS as a set of stateless EC2 nodes controlled by an ASG, backed by an RDS instance. I need to throttle the overall number of outgoing messages my service sends but can'...
2
votes
1answer
361 views
AWS SQS FIFO Limit Workaround
According to the AWS feature page for SQS:
FIFO queues support up to 300 messages per second
Standard queues support a nearly unlimited number of transactions per second (TPS) per API action.
...
0
votes
1answer
55 views
The right way of doing images manipulation in a serverless app
This question is not specific to images, but this is the current issue im having so im using this as an example.
So basically, I have some kind of a pipeline that some data(in this case, an image), ...
3
votes
1answer
126 views
Sharding and application routing cross region on AWS
We current have a very simple Multi Tenant monolith, with a SQL SERVER backend (Self hosted on EC2 on AWS), and multiple application services talking to one DB behind an Classic AWS ELB. Our database ...
2
votes
0answers
263 views
Getting AWS SNS Messages into Azure Service Bus
I have a similar question to one already asked about SQL to Azure Service Bus - https://stackoverflow.com/questions/51650073/consume-aws-sqs-and-publish-to-azure-service-bus. I want to get messages ...
-2
votes
1answer
82 views
How to start designing a complex information system?
Recently I've been assigned to a new project to start with designing and visualizing business processes. Unfortunately I just can't wrap my head around this one, I understand what the client wants but ...
1
vote
1answer
138 views
What's stopping Cloud Platforms from owning the whole deployment process?
Although things have come a long way in recent times, in my workplace we're still talking about DLLs, containers, configs, substitutions, pipelines - all complexities when it comes to shipping code.
...
0
votes
1answer
101 views
How to route/implement port forwarding with AWS/Apache web server (httpd) with a Node.JS app on port 3000?
I'm trying to understand this and having a hard time. I usually use Ubuntu server, with Apache2, and this time I am not. So I'm a bit out of my element. I'm also a software developer and I don't do ...
1
vote
1answer
962 views
Serverless event-sourced architecture using AWS offerings
This is my first post on the Software Engineering Stack Exchange so let me know if something is wrong with it.
I'm looking into the serverless offerings of Amazon to try to figure out if that is the ...
0
votes
2answers
136 views
How to build reports for an ecommerce webapp without slowing it down
We have a medium sized ecommerce website which is entirely custom built in using Java and MySQL database on AWS infrastructure. Over a period of past several years, our orders volume has grown ...
0
votes
1answer
315 views
FTP upload on AWS Elastic Beanstalk
We have our REST API running on AWS Elastic Beanstalk (NodeJS/Linux), there are some legacy devices that their only way to communicate with the world is uploading a file using FTP.
These devices ...
0
votes
1answer
165 views
Co-relating messages in Kafka
Context
We are building an application that acts like a hub for messages coming in from multiple Producers and it assigns to Consumers based on certain criteria.
Need is to have the design highly ...
0
votes
2answers
437 views
Preventing abuse of API which is called via client side Javascript
I'm setting up an AWS API Gateway + Lambda micro-service that will return non-sensitive but user-specific data.This means that the responses from the service can't be cached and served to multiple ...
0
votes
1answer
107 views
Watermark on Video implementation , Server or Client
Below is the context,
Right now i'm developing an android application, which will allow the user to upload the video and image to the server. I want to allow the users(his friends) to download the ...
0
votes
0answers
73 views
aws lambda service for name generation
I want to implement a lambda service that would provide me random names (generated using patterns, some hardcoded values and so on) by GET request.
For name generation I just use the set of prepared ...
0
votes
2answers
542 views
Send multiple record types to a single Kinesis stream or not?
We have a bunch of services which are using AWS Kinesis to communicate via events. We have different events such as MasterDataUpdatedEvent, PriceUpdatedEvent or AvailabilityChangedEvent.
Currently, ...
1
vote
1answer
127 views
Building a real-time API on a Kafka/Kinesis-centric structure
I am a beginner to the event-driven data architecture using Kafka / Kinesis as the centrepiece (currently Kinesis) and I have some questions regarding how to build a WebSocket API on such a structure. ...
0
votes
1answer
547 views
Building docker images in Gitlab CI and pushing to AWS ECR.
I want to do something fairly simply in my Gitlab CI pipeline - I want to build the docker image and push it to an AWS ECR repo.
I've looked at this Gitlab CI guide and one of the suggestions it ...
2
votes
1answer
1k views
Does AWS Kinesis guarantee a specific type of ordering?
From AWS documentation:
A Kinesis data stream is an ordered sequence of data records. Each record in the stream has a sequence number that is assigned by Kinesis Data Streams
I don't see any ...
0
votes
0answers
109 views
Is MapReduce a correct framework for Extract, Transform, Load of data?
EDIT
I am working on a project to update a legacy ETL infrastructure that supports a number of clients, each with a slightly different setup.
Constraints that cannot be changed:
Source data can ...
1
vote
0answers
19 views
What's the preferred way to section a dynamodb table by client?
Working on an application that uses DynamoDB for data storage, which is new-ish for me. There are two tables, both use a simple guid for their key. I need to divide data per client. Someone can sign ...
2
votes
1answer
621 views
How to automate my AWS spark script
I am new to AWS and i have learnt and developed code in spark -scala .
My application basically merge two files in spark and created final output.
I read both files (MAIN files and INCR files )in ...
-2
votes
1answer
66 views
GNU license and hosting source code
Currently working with a client and their lawyers to launch a web application.
The website will be hosted in AWS on Debian Linux instances.
Debian contains (among others) the GNU General Public ...
4
votes
2answers
830 views
Which is the best option to exchange large amount of data in a micro-service architecture?
Application I am working on requires text contents to be extracted from various proprietary document formats like Microsoft word documents (doc, ppt, xls ), pdf and etc.
I am planning to implement a ...
1
vote
0answers
71 views
iOS Push Notifications Certificate Sync Cert to Servers
I'm not sure if this is the correct form to be posting this on Stack Exchange. If not I apologize in advance and hope someone can point me in the right direction of where to post.
I have a AWS Load ...
0
votes
2answers
672 views
How can I create a Python microservice on AWS that both accepts REST connections and processes SQS messages?
Background: I'm working on a new project at work that will run in AWS. We're trying to use a modern microservice architecture and take advantage of cloud technology, but we don't have much experience ...
1
vote
2answers
379 views
NoSQL or SQL for my situation? [closed]
I'm sure this has been asked multiple times but for me it's more specific as I know what I'm building. I wrongly posted this to stack-overflow (now removed) so I'm hoping this is the best place to ask....
1
vote
2answers
638 views
Best practices in saving user data when using identity provider like AWS Cognito
So in my mind, the usual users table that we are used to having, is going to split in two:
whatever data is within the Identity Provider
the domain specific data (user.facialHairStyle) is going to ...
0
votes
1answer
541 views
How can you reuse code for similar functions on AWS Lambda?
I have 3 Lambda functions that do very similar things: scrape data from different sections of a website.
Each function returns a collection of objects that have similar properties. The method of ...
-1
votes
1answer
63 views
Amazon RDS read replica without support from application [closed]
I'm very new to this,so please correct if I'm wrong.
I've understood that with Amazon RDS one can create read replicas and use those for all read-only operations. This helps scaling the application a ...
7
votes
2answers
2k views
DB migration strategy for docker containers in AWS ECS
I am working on creating a set of microservices in Docker (using .net core and AWS RDS as the backbone, but that is not relevant).
As part of the deployment, the old container and new container co-...
1
vote
1answer
762 views
Messaging between microservices on AWS
Here is the scenario:
We are running a microservices style architecture on AWS using API Gateway and EC2 instances. Let’s say there are five services: Photo Service, Topics Service, User Posts ...
2
votes
1answer
297 views
HA for two availability zones in AWS
I am trying to setup consul for service discovery. We have hosted our infrastructure on AWS Mumbai region (We use only one region). I was going through the consul documentation where it is mentioned ...
-1
votes
2answers
120 views
Autogeneration cloudformation. Good or bad
at my company we are using AWS Cloudformation to manage our infrastructure. The templates are written in JSON. The problem is that the templates are getting bigger by the day 5k Lines. Company policy ...
4
votes
2answers
605 views
Where should authorization live in an application using API gateway/lambda and openid?
For an application which:
Will use Google OpenID for account creation & authorization
I plan on allowing any user to "create an account" which is why I want to use Google for this
AWS API ...
