6 votes
Accepted

Server architecture for short bursts of ~150 parallel CPU-heavy subqueries

Since you are CPU-limited, you need to get your hands on 150 CPU cores, one for each thread. This rules out a single server, since a server of such proportions would be prohibitively expensive – and ...
amon's user avatar
  • 134k
3 votes

Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?

I believe there is a 6Mb limit on the request/response, which would make it unsuitable. https://docs.aws.amazon.com/lambda/latest/dg/limits.html Ideally you want a resumable stream of data for large ...
Ewan's user avatar
  • 74.6k
3 votes

How to efficiently process CPU intensive tasks on the server in the background

Rails has a built-in framework for background task processing, so lengthy tasks don't block the controller from returning a response to the client. It's called Active Job. (See: https://guides....
Alex D's user avatar
  • 1,308
3 votes
Accepted

Messaging between microservices on AWS

"We are considering creating a queue for each microservice and publishing the messages to the queues via SNS." Yes, then your queues can subscribe to the notification and you no longer need the ...
Ewan's user avatar
  • 74.6k
2 votes

Server architecture for short bursts of ~150 parallel CPU-heavy subqueries

I would design my interface and task master under two assumptions: Processing power is distributed I don't always have enough processing power available If the jobs are inherently long, and the ...
svidgen's user avatar
  • 14.6k
2 votes
Accepted

What is the threat model for deciding between unencrypted vs. encrypted EBS volumes?

Basically, you've answer you're own question. Except one nagging thing... all hard drives fail at some point, and must be disposed of. If someone disposed of the drive improperly, or simply steals the ...
Greg Burghardt's user avatar
1 vote

When creating a RESTful API in AWS, why would you use EC2 over a Lambda integration?

Why EC2, some points: More control EC2 allows for more control on what is running on the hardware like specific tech versioning etc. EC2 allows for custom provisioning. Lambda is more hands off ...
Jon Raynor's user avatar
  • 11.4k
1 vote

Allowing a dynamic list of IP addresses to access a specific port in an AWS EC2 Instance

Based on the tools mentioned in your post, one potential way to solve this is through the use of Lambda function. It might be overkill for your situation but you could have a function determine the ...
ServerMonkey's user avatar
1 vote
Accepted

How to automate my AWS spark script

You have 2 options. If you also want to shutdown and start cluster then option 1, the AWS Data Pipeline is better: 1. AWS Data Pipeline DeveloperGuide: what-is-datapipeline 2. Oozie Workflow You ...
Rene B.'s user avatar
  • 170
1 vote
Accepted

Upload fingerprint from android/ios to cloud

I don't know the details for Android, but iOS will most definitely not allow you to directly access the fingerprint data, let alone upload it. By handling all biometric encryption/decryption in a ...
Kris Van Bael's user avatar
1 vote

Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?

If you know the limits on the response then you can use aws lambda but also make sure you know how to handle the situtation when aws lambda timed out because of reasons out of your control. How would ...
Ankur Kothari's user avatar
1 vote
Accepted

Are AWS Lambda functions a good fit for the use case of fetching huge amount of records from database on User request?

Sounds like a possible use case but it really depends on the type of database, how much data you're returning and if the hard limits on AWS services prohibit the use of API Gateway/Lambda. SQL ...
Fernando's user avatar
  • 126

Only top scored, non community-wiki answers of a minimum length are eligible