0

Firstly, I want to state that this is my first time trying to build an app. I am doing this for fun, mostly as an exercise to learn system design/architecture. The goal is to build an app that would be scalable to millions of users (not that this app will actually have millions of users-- its just to build my own understanding of large scale distributed systems).

What I am using: AWS architecture tools - Lambda, DynamoDB, Amplify

What I would like to build: I downloaded and imported Movies from IMDB into DynamoDB. The goal is to allow the user to search for a movie and display its info from the DB. The user should also be able to rate the movie.

What I am asking:

  1. Which AWS tool should I use for searching the DynamoDB?
  2. Alternatively, should I be using a different database to search from? If I understand correctly, AWS ElasticSearch is actually a database. Should I just be using this instead?

Please also let me know if you have any general advice, or if I am thinking about all of this incorrectly.

3
  • 1
    I am not sure why someone downvoted this question. I think that I phrased my question in an exact manner and there is an answer to it. If you think it needs clarification or something else, please let me know and I can update the question. If I cannot ask this here, then where should I ask it? In other words, don't be a dick. Feb 10, 2021 at 20:50
  • DynamoDB is a key-value store and is not designed for search. See docs.aws.amazon.com/cloudsearch/latest/developerguide/…
    – Dan Wilson
    Feb 10, 2021 at 21:25
  • There are multiple valid solutions. Which one is best depends on your specific requirements.
    – Dan Wilson
    Feb 10, 2021 at 21:26

1 Answer 1

3

Which AWS tool should I use for searching the DynamoDB?

You don't. DynamoDB is designed for quick and efficient lookup by key and by key only1. If you are attempting to search DynamoDB as any part of "normal operations" then it means you're misusing it. It sounds very much like DynamoDB is not the right tool for what you want to do - go back and look at the process that ended up with you choosing DynamoDB, and then re-evaluate your process as the process came to a bad conclusion.

Alternatively, should I be using a different database to search from?

Yes. Although you should probably consider just using whatever database you decide on for everything, and dumping DynamoDB entirely.

If I understand correctly, AWS ElasticSearch is actually a database. Should I just be using this instead?

We can't tell you this as we don't know what your requirements are. I would just start with a good old SQL database and see how far you can get with that.

Stepping back a bit:

I am doing this for fun, mostly as an exercise to learn system design/architecture.

Honestly, it doesn't sound like you are; it sounds more like you have seen a bunch of shiny things on the AWS website, have decided you want to use those, and are now trying to force a problem to fit into those technologies. That is a perfectly fine thing to do if you want to learn how to use those technologies, but it is not system design. For system design, you start with a bunch of requirements, look at the options and choose the most appropriate one, even if it is a plain old SQL database.

1. Or by secondary index, but that's effectively just a second key.

3
  • Thanks for your response. I think that you are correct. I am trying to use some "shiny" things on the AWS website and force it to fit an application.The reason is that as an ML Engineer, I now have to sometimes do large scale/ distributed System Design interviews amidst the 5 other types of interviews that I already have to do depending on the company (Coding, Large Scale ML Design, Behavioral, Stats/Probability etc). I recently applied to FB and had an on site interview. According to my feedback, I did well in all interviews except the large scale system design one. Thats why I am doing this. Feb 11, 2021 at 1:55
  • I have already tried reading the books and doing other online courses, however it doesn't really sink in unless I have personal experience. You might think it sounds dumb to randomly want to simulate building a huge app intended for millions of people when that will never occur, but so far I cannot figure out a better way to learn how to design large scale systems. This is why I wanted to use DynamoDB. The interviewers are always big on NoSQL. I think that designing an app to search in a large scale manner is a good enough idea to simulate one of these interview questions. Feb 11, 2021 at 1:59
  • Coming up I have on site interviews with Google and Amazon. I intend develop a better intuition for large scale sys design before those interviews. It is unfortunate though that they now require ML engineers to know about distributed systems and excel in the interview as much as another type of software engineer since our specialty is the ML part of it, and still work with other engineers whose job it is to know more about this and not about ML Feb 11, 2021 at 2:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.