0

I'm in the process of writing an app that will ultimately display analytics to the user. I've written a service that collects data from an API. This data will then be processed, stored, then when the user requests the data, pulls it from the store and displays it. Fairly straightforward. We plan on using MongoDB for the app database (storing users, settings, etc.). I've read that Redis is good for storing metric information because of the key/value pair nature.

My question is, what would be the best way to go about interchanging how the data comes from the API service to the user being able to request it? I've initially come up with storing the API data in another MongoDB store, seperate from the app. Then having another service that runs at a longer interval than the API service that aggregates the raw data in Mongo, moves it into Redis, then archives the parsed Mongo data into either some log file or something. The app would then be able to reach into Redis to grab the metrics based on predetermined keys.

Is Redis even the right option for something like this? I've also considered swapping out MongoDB with Postgres or MySQL since operations like SUM run well on a relational platform.

  • 2
    I don't have enough information to answer your question, but I do remember that Redis was originally built to be the backend for an analytics system. So on the face of it, it seems like a good choice to use in the backend of an analytics system. – Racheet Jun 18 '13 at 16:47
2

I'm not sure how much redis will help here -- it can do some fancy tricks for sure, but you probably need to work out getting this thing built and running and have some baseline performance before sticking in a caching layer. Personally I would start with just using MongoDb's map reduce functions and take it from there.

  • That makes sense. People do say that you should optimize later. Would it still make sense to have two seperate Mongo databases, one for the raw API data to be processed, and the other for the app? – Justin Chmura Jun 18 '13 at 20:12
  • Perhaps -- but I really don't have enough seat time with mongodb to know if separating the databases makes sense or not. Based on experiences with other document DBs I would say that you would be better off building a bigger cluster than a separate server. – Wyatt Barnett Jun 18 '13 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.