How should I store time stamps so that it's easily accessible and won't take too much space

Question

tldr at the bottom if you don't want to read all this! :)

First of all the db I'm using is MongoDB! So I've been building a fun project and all has been well but I hit a small problem. Effectively, the project could be thought of as a simple button on a website. For each click of the button, the click will be sent to the back end via sockets and recorded and stored in a database. This is very small and easy to do as I just have a simple Mongo database with a key called "buttonClicks" that increment by 1 each time it's clicked. There's also some other arbritrary incremental data sent as well like for examples sake, the browser of which the click was sent from. Regardless, the schema currently looks like this

{
    name: Admin
    buttonClicks: 22,
    whichBrowser: {chrome: 10, edge: 12}
}

As you can see, this is all incremental data, and since there's only a handful of browsers on the planet, the whichBrowser object won't get that big.

The problem is I also need a time stamp. What I mean is the way it's setup now, I just know how many times the button was clicked from day 1 till now. What if I want to see how much button clicks there was today/week/month. I plan to show this in a graph so I would also like to bin it by X amount as well. So I may want to graph it where it shows me the clicks through the past day in 5 minute intervals, or clicks through the past week in 10 minute intervals, etc.

How can I achieve this? As it is right now my database is very small since it basically has 1 key with a value that simply increments, but if I were to store the time stamp as is for each click, I imagine the database will get MASSIVE with the "buttonClicks" key having a value of a huge object, something like

{
    name: Admin
    buttonClicks: [{
        date: "3/27/2020 - 12:44:15",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:15",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:15",
        browser: edge
    },{
        date: "3/27/2020 - 12:44:16",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:16",
        browser: fire fox
    }]
}

As you can see, only 5 clicks were recorded but it's already so big. Not to mention (in this example) only 2 seconds has passed and this array already has 5 objects.

Thank you! So does anyone have any advice on how to can store each click (and other info like browser)

tldr: A button is clicked on a site and recorded in the back end. It takes little space for now since it's just incremental data (just up the value by 1 when a button click is recorded) but I realized I need the date and time as well for data visualization purposes. How can I record the data in a way for each click so it won't take a lot of space and I can easily manipulate it (like how much clicks were there the past week, past 10 days, etc)

Answer 1

Your main problem here is not the anmount of data (mongo can handle BIG stuff) but the maximum document size of 16 MB.

Thus, you need some kind of "paging" like a log-rotation on a file system. A standard approach is, that you start off with a document per given time interval, e.g. a day:

{
    "name": "Admin",
    "recordingStart": "3/27/2020",
    "buttonClicks": [...]
}

In many cases, this will be enough already.

But maybe your site is high-traffic, so if this grows too large, add a continuation document, i.e.

{
    "name": "Admin",
    "recordingStart": "3/27/2020",
    "ordinal": 1,
    "buttonClicks": [...]
    "continueWith": 2
}

{
    "name": "Admin",
    "recordingStart": "3/27/2020",
    "ordinal": 2,
    "buttonClicks": [...]
}

With a decent base time interval, this can also ease your later aggregations a lot.

Answer 2

Maybe borrow some textbook "tricks" from compression algorithms?

{
    name: Admin
    buttonClicks: [{
        date: "3/27/2020 - 12:44:15",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:15",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:15",
        browser: edge
    },{
        date: "3/27/2020 - 12:44:16",
        browser: chrome
    },{
        date: "3/27/2020 - 12:44:16",
        browser: fire fox
    }]
}

Consider a dictionary:

startDate: "3/27/2020 - 12:44:15"
chrome: 1
edge: 2
firefox: 3

And use the convention that seconds represents seconds passed since startDate.

Your original data then becomes:

{
    name: Admin
    buttonClicks: [{
        seconds: 0
        browser: 1
    },{
        seconds: 0,
        browser: 1
    },{
        seconds: 0,
        browser: 2
    },{
        seconds: 1,
        browser: 1
    },{
        seconds: 1,
        browser: 3
    }]
}

Does this save some space?