I have a application where user purchases/click the certain products. I need to design the click stream analysis here which product got clicked how many number of time, user/geographical detail click those product
Here is the design i am planning
- Whenever user clicks the product , generate the async call to produce the message on queue
- I am planning to use the distributed message broker and selected the Kafka for its distributed model and performance. My load is 100 to 150 request per second.
- Use Apache spark streaming module to parallely process the message from queue and store it in document based DB i.e. MongoDB. I selected the Mongo as i don't have relational requirement here and probably down the line i may use Sharding once data becomes big.
Let me know if my design/technolgy stacks looks good ? If not where it can be improved.