I would like to generate analytics on a per-user basis on how many times they've viewed a particular page for 7 days, and how many times they've viewed a page during their lifetime as a user.
This will be tracked in 1d, 7d, 14d, 30d, and lifetime intervals. If a user visits the page today, they will appear to have 1 visit in the 1d, as well as 1 additional visit in all date range categories above 1d- since visiting once today means you've visited at least one in all other date range categories.
To do this I am storing all events in a data lake, and rolling up these date range counts every 24 hours based on the criteria. The actual users data is in a document, but it wouldn't be feasible to store all events for a user in their document given how much data that would eventually be, and the inevitable data skew that would create on the cluster. This works now, but it's taking more and more time to generate these rollups as the data grows, and even with partitioning, the number of pages we are doing this rollup on is growing at a pace where the current method may not scale gracefully.
When we receive these events, we could update the users counts at runtime. But without the context of the date, there would be no "dropoff". If it were a lifetime count we could always increment that field, but the counts need to update daily as a page view today is not a page view tomorrow.
Something that stands out to me is that the counts will consistently gravitate towards one end. But that may just be an observation and not anything useful.
Is the only way to do this the way I've described, or is there a more clever way to update these fields?