I'm trying to create a website as a forum similar to Reddit, but on a smaller scale (no comments also). The website will allow users to post a link to a forum. Then they can view a list of New or Trending links based on upvotes (gathered from an external API).
So basically all I need is to have a REST API to post links to one of many long lists, each list then users can view from with GET requests. There also needs to be a quick way to get the last 7 days of links so that the most popular ones can be computed. I'm accommodating for a maximum of about 100,000 links posted daily (about 1 per second) or and maybe 10 requests to Trending and New posts.
My current architecture would be:
- Stores past n (likely n=7) days of links in memory, then periodically empties that list into a long-term storage file.
- Then to get New links, generally just queries memory but if a subforum is not very active gets older links from file.
- Periodically updates the list of Trending links (short list in memory) for each subforum, then to get these Trending links queries that list in memory.
This will be mostly built for learning and it will not reach the 100,000 links daily, but I want to build something that can scale to that level so that I will be prepared if I have to in the future.
Would my above architecture be scalable, is there a better way of doing this, any other suggestions? Thanks for your help ahead of time.