I've been a heavy user of "batch jobs" to perform different kinds of logic in systems. Over the last couple of weeks, I've been thinking and reading about other approaches and I wanted to bounce some of my ideas and get some feedback and trips.
So to set the stage, let's look at a use case:
We're creating a chat application that users can embed on their websites. When a user creates a new account in the system it should be running as a "trial" account for 30 days. Five days before the trial ends we should send an email reminder if the user hasn't configured payment. If the trial period ends and the user still has not configured payment, we should notify the user. When the service has been inactive for 2 days we want to check if the user is trying to use the chat-script on their website and notify them again if they do.
So the "batch approach" to this could maybe look something like this:
Create a batch-job that fetches all users that have 5 days left of the trial. Check conditions, make sure we haven't already sent a reminder and send a reminder if needed.
Create another batch -job that looks for users where the trial period has passed, change the status to "inactive" if payment has not been configured. Make sure we haven't already sent a reminder and send a reminder if needed.
Create a batch-job that looks for users that have been inactive for two days, check if they are still using the script on the website, check that we haven't already notified the user and send notification if needed.
While this probably works fine I've been thinking about another approach to model these kinds of "future events" with the goal to make the process more explicit, easier to follow and easier to debug.
My current idea looks something like this:
When a new user is created we already know the timeline that we have for the events in the future. We could store some rows in a database to indicate the events we know that we might want to fire in the future. So, when the user is created we also queue these tasks:
- 25 days from now we want to send a reminder if payment has not been configured.
- 30 days from now we want to check the payment-status and inactivate the account if needed. At this point, if the account is inactivated we can schedule the follow-up task to check if the chat-script is still used on the users website two days later.
If the user at any point during the first 25 days configured payment we could either remove the upcoming "tasks" or set some status to ignore them. Same thing if the user decides to close the trial.
I can see some clear benefits of this:
Each "task" represents just one thing that needs to be done. No need to handle problems that comes with batch-jobs like "what if it blows up in the middle of the processing?"
We have a clear overview of "upcoming events" that we can query both on a wide "system level" and on the user-level.
We could store task-execution result on the queue item and set it to "done", this way we have a log of each thing that happened.
The concept is of cause reusable to many types of entities, not just users.
In terms of implementation, I'm still not sure how to implement this. I've been reading about distributed messaging systems like e.g. RabbitMQ or Azure Service Bus which obviously is great in terms of performance but I see a lot of value in the possibility to query the queue and sorting and also to keep a lot of events. A "poor man's solution" could be to pull the database for upcoming tasks and execute them and I guess that the sweet spot might be a combination where the details are stored in the DB and the queue is used for scheduled execution.
I would love to get some feedback here.
- Do you think that the "new approach" makes sense or would you argue against it? Why?
- How would you solve a similar use case?
Thank you so much for taking the time to read this and for any feedback!