Questions tagged [aws]

AWS refers to Amazon Web Services.

Filter by
Sorted by
Tagged with
2 votes
2 answers
1k views

Preventing abuse of API which is called via client side Javascript

I'm setting up an AWS API Gateway + Lambda micro-service that will return non-sensitive but user-specific data.This means that the responses from the service can't be cached and served to multiple ...
user avatar
  • 87
0 votes
1 answer
355 views

Watermark on Video implementation , Server or Client

Below is the context, Right now i'm developing an android application, which will allow the user to upload the video and image to the server. I want to allow the users(his friends) to download the ...
user avatar
  • 381
0 votes
0 answers
77 views

aws lambda service for name generation

I want to implement a lambda service that would provide me random names (generated using patterns, some hardcoded values and so on) by GET request. For name generation I just use the set of prepared ...
user avatar
  • 203
0 votes
2 answers
2k views

Send multiple record types to a single Kinesis stream or not?

We have a bunch of services which are using AWS Kinesis to communicate via events. We have different events such as MasterDataUpdatedEvent, PriceUpdatedEvent or AvailabilityChangedEvent. Currently, ...
user avatar
1 vote
1 answer
261 views

Building a real-time API on a Kafka/Kinesis-centric structure

I am a beginner to the event-driven data architecture using Kafka / Kinesis as the centrepiece (currently Kinesis) and I have some questions regarding how to build a WebSocket API on such a structure. ...
user avatar
1 vote
1 answer
926 views

Building docker images in Gitlab CI and pushing to AWS ECR.

I want to do something fairly simply in my Gitlab CI pipeline - I want to build the docker image and push it to an AWS ECR repo. I've looked at this Gitlab CI guide and one of the suggestions it ...
user avatar
  • 2,463
5 votes
1 answer
5k views

Does AWS Kinesis guarantee a specific type of ordering?

From AWS documentation: A Kinesis data stream is an ordered sequence of data records. Each record in the stream has a sequence number that is assigned by Kinesis Data Streams I don't see any ...
user avatar
  • 277
0 votes
0 answers
669 views

Is MapReduce a correct framework for Extract, Transform, Load of data?

EDIT I am working on a project to update a legacy ETL infrastructure that supports a number of clients, each with a slightly different setup. Constraints that cannot be changed: Source data can ...
user avatar
1 vote
0 answers
21 views

What's the preferred way to section a dynamodb table by client?

Working on an application that uses DynamoDB for data storage, which is new-ish for me. There are two tables, both use a simple guid for their key. I need to divide data per client. Someone can sign ...
user avatar
  • 111
2 votes
1 answer
1k views

How to automate my AWS spark script

I am new to AWS and i have learnt and developed code in spark -scala . My application basically merge two files in spark and created final output. I read both files (MAIN files and INCR files )in ...
user avatar
  • 131
-2 votes
1 answer
93 views

GNU license and hosting source code

Currently working with a client and their lawyers to launch a web application. The website will be hosted in AWS on Debian Linux instances. Debian contains (among others) the GNU General Public ...
user avatar
6 votes
2 answers
5k views

Which is the best option to exchange large amount of data in a micro-service architecture?

Application I am working on requires text contents to be extracted from various proprietary document formats like Microsoft word documents (doc, ppt, xls ), pdf and etc. I am planning to implement a ...
user avatar
1 vote
0 answers
102 views

iOS Push Notifications Certificate Sync Cert to Servers

I'm not sure if this is the correct form to be posting this on Stack Exchange. If not I apologize in advance and hope someone can point me in the right direction of where to post. I have a AWS Load ...
user avatar
0 votes
2 answers
1k views

How can I create a Python microservice on AWS that both accepts REST connections and processes SQS messages?

Background: I'm working on a new project at work that will run in AWS. We're trying to use a modern microservice architecture and take advantage of cloud technology, but we don't have much experience ...
user avatar
  • 3
1 vote
2 answers
499 views

NoSQL or SQL for my situation? [closed]

I'm sure this has been asked multiple times but for me it's more specific as I know what I'm building. I wrongly posted this to stack-overflow (now removed) so I'm hoping this is the best place to ask....
user avatar
2 votes
2 answers
1k views

Best practices in saving user data when using identity provider like AWS Cognito

So in my mind, the usual users table that we are used to having, is going to split in two: whatever data is within the Identity Provider the domain specific data (user.facialHairStyle) is going to ...
user avatar
  • 121
1 vote
1 answer
731 views

How can you reuse code for similar functions on AWS Lambda?

I have 3 Lambda functions that do very similar things: scrape data from different sections of a website. Each function returns a collection of objects that have similar properties. The method of ...
user avatar
-1 votes
1 answer
82 views

Amazon RDS read replica without support from application [closed]

I'm very new to this,so please correct if I'm wrong. I've understood that with Amazon RDS one can create read replicas and use those for all read-only operations. This helps scaling the application a ...
user avatar
8 votes
2 answers
3k views

DB migration strategy for docker containers in AWS ECS

I am working on creating a set of microservices in Docker (using .net core and AWS RDS as the backbone, but that is not relevant). As part of the deployment, the old container and new container co-...
user avatar
  • 364
2 votes
1 answer
1k views

Messaging between microservices on AWS

Here is the scenario: We are running a microservices style architecture on AWS using API Gateway and EC2 instances. Let’s say there are five services: Photo Service, Topics Service, User Posts ...
user avatar
2 votes
1 answer
413 views

HA for two availability zones in AWS

I am trying to setup consul for service discovery. We have hosted our infrastructure on AWS Mumbai region (We use only one region). I was going through the consul documentation where it is mentioned ...
user avatar
-1 votes
2 answers
177 views

Autogeneration cloudformation. Good or bad

at my company we are using AWS Cloudformation to manage our infrastructure. The templates are written in JSON. The problem is that the templates are getting bigger by the day 5k Lines. Company policy ...
user avatar
4 votes
2 answers
690 views

Where should authorization live in an application using API gateway/lambda and openid?

For an application which: Will use Google OpenID for account creation & authorization I plan on allowing any user to "create an account" which is why I want to use Google for this AWS API ...
user avatar
  • 12k
1 vote
1 answer
906 views

Integrate an external rest api with my web application

I have a service running on AWS exposed via a rest API. The service processes files. Clients of the service post a file to the api and are returned a Job ID. Clients can then request the status of ...
user avatar
3 votes
0 answers
378 views

Examples for "serverless" Java service for AWS Lambda

Doesn anybody have practical experience with implementing a (micro)service based on AWS Lambda? My team is currently working on an MVP. We started with a monolith and that has worked very well. It's ...
user avatar
  • 489
2 votes
1 answer
424 views

Cost of Cloud Vendor Lock-In

Consider a greenfield development situation where cloud tools are being considered vs. in-house solutions, in the vein of AWS SQS vs. self-hosted Kafka, ECS vs. Mesos-Marathon, Lambda/Azure Functions ...
user avatar
  • 3,108
2 votes
0 answers
481 views

Designing large nanoservice architecture with AWS Lambdas

I'm moving from a coupled architecture to a decoupled architecture using microservices with AWS Lambda. Here is my current architecture: Each API Gateway route is linked to a specific Lambda, each ...
user avatar
  • 169
-1 votes
1 answer
180 views

AWS CLI > why is the JSON output indented?

Just reading through the AWS CLI docs, and I've noticed that all JSON output is indented (i.e. "pretty" formatted)... Doesn't this add a disproportionate amount of bloat to the responses? If so, why ...
user avatar
6 votes
3 answers
297 views

Server architecture for short bursts of ~150 parallel CPU-heavy subqueries

The client sends a query (a few hundred characters) to the web service. This query can be split into 20 to 150 subqueries with a simple regex. Those subqueries can then be computed independently and ...
user avatar
  • 163
1 vote
2 answers
1k views

How to design authorization between docker microservices?

I have 2 micro-services. One is a Django API, the other is a worker. Each one of these lives in side a docker container. The API is a public API for the web front end. It handles CRUD operations for ...
user avatar
2 votes
1 answer
1k views

Deliver message to websocket client connected through load balancer

The problem I have a number of clients connected via websocket to nodes of my web application through a load balancer. What I need is to deliver notifications on a per-user basis. My idea My idea ...
user avatar
0 votes
1 answer
59 views

When is cloud BlobStorage used?

Let's say we have an app deployed to Azure. App has external files not bundled with it, files which are required for app to operate. They are seperate but are essentially part of the software. ...
user avatar
  • 1,129
3 votes
0 answers
783 views

How do I implement Socket.io Leaderboard using AWS Dynamo DB cost-effectively?

Every time a user submits a score, Socket.IO causes all the users get their leaderboard updated. This causes every single user connected through Socket.IO to read from the Dynamo DB database (every ...
user avatar
  • 31
0 votes
1 answer
503 views

Securing Amazon DynamoDB from Over-the-web attacks

I'm in the process of building out an architecture that will heavily leverage DynamoDb. My organization is considering the possibility of storing sensitive data (config info) in DynamoDb, as well as ...
user avatar
3 votes
2 answers
282 views

Serverless architecture: How to deploy changes in common business logic?

I've been hearing lately more and more about serverless architectures which are based on products such as AWS Lambdas, Azure Functions, Google Cloud Functions, etc. I understand the advantages of ...
user avatar
6 votes
0 answers
360 views

Rails on AWS Elastic Beanstalk - limitations

I have been looking at AWS Elastic Beanstalk service for the deploymeny of my rails application, and I came to think it was actually a bad idea given some specific requirements of my application. But ...
user avatar
0 votes
1 answer
949 views

Most elegant way to execute a AWS lambda function with preloaded parameters on a specific time

I am looking for the following workflow: Add an entry to a table of my Postgres database with a end_date and state (with the following values: active or done) columns. Execute a lambda at the exact ...
user avatar
  • 169
1 vote
2 answers
6k views

Best way to manage Git branches in different environments

I had a basic question regarding using a git repository in different environments. I'm more of a Sys admin than a web developer, so the teams I've worked on before just worked right in the master, ...
user avatar
  • 119
1 vote
0 answers
136 views

Distributed Design to Speed Up Video Processing

Currently I have one web server running Django, which takes a series of images from a user, uploads them to an Amazon S3 bucket, and finally calls an Amazon Lambda instance, which runs a python script ...
user avatar
1 vote
1 answer
843 views

Amazon Echo Development on LAN

I have followed the tutorial from Amazon to get started with the Echo. I made a skill and setup an application server on their AWS Lambda for basic testing. I have a few questions about the Echo, ...
user avatar
2 votes
1 answer
397 views

Updating a video's view count. Should this happen client-side or server-side?

I built an iOS app similar to Vine, where a user can scroll a feed of videos that auto play. All of these video have a View_Count field in the backend to keep track of how many times the video has ...
user avatar
0 votes
0 answers
51 views

What are the best practices of scaling small scale webservices in AWS?

I've deployed an entreprise app in AWS, in the first time i was hosting it in t2.micro, the app is intensivily used in 15 minutes a day, with some light queries in work hours during the day, after ...
user avatar
  • 101
-1 votes
1 answer
439 views

How to connect mean.js to AWS S3

I am trying see how I can upload images and file to aws S3 via the mean.js framework. I am not sure how the backend is linked to S3. There are no resources online about this specific framework and ...
user avatar
-1 votes
1 answer
676 views

Is there any harm in using amazon AWS as a development environment? [closed]

What modifications do I need to make to my workflow, and which skills do I need to learn in order to develop with Ruby On Rails on a Linux cloud server on Amazon Web Services (AWS)? In particular, I ...
user avatar
2 votes
1 answer
99 views

Will hosting documents on AWS reduce our load?

Due to partnerships, we are required to host our website through our partner company. However, we had an idea that perhaps we could offload the document storage to an AWS document server. The ...
user avatar
  • 429
1 vote
1 answer
2k views

Implementing distributed Semaphore for AWS workload

We have a messaging gateway in AWS (Amazon Cloud) that comprises of a number of nodes (Linux servers) responsible for receiving, routing, despatching and escalating messages (emails, SMS' and voice ...
user avatar
  • 111
0 votes
1 answer
711 views

Backing up messages in S3 within a Storm topology

In my project we are trying to build up a KIND-of-a-lambda storm based architecture. The component would be responsible for indexing the site usage events so we expect a quite massive random load. The ...
user avatar
7 votes
1 answer
7k views

Best way to notify the client in real time that their queue (e.g. SQS) job has finished?

Current Scenario Our application allows users to upload (Amazon S3) and manage their files through our interface. Currently those users can download the files directly from S3/Cloudfront through our ...
user avatar
6 votes
4 answers
7k views

Building a distributed system on Amazon Web Services

Would simply using AWS to build an application make this application a distributed system? For example if someone uses RDS for the database server, EC2 for the application itself and S3 for hosting ...
user avatar
  • 6,528

1 2
3