Is there a best practice for scheduling forecast notifications?

Question

Let's say we write an application that can use multiple forecasts from different sources. It does not really matter what type of data we are talking about, ofc. it can be weather forecast, stock price forecast, and so on... The main thing here, that we can have multiple time periods, for example .5h, 3h, 9h, 1d, 3d, 30d, etc... Usually the longer the period, the less accurate the forecast is. What I struggle with that I don't know what is the best practice by sending notifications.

It would be nice to send a notification as early as possible, but it would be bad to make mistakes and send out false notifications. For example I could notify the user 3 days before the predicted event, but 1 day before it I could cancel the whole thing, because the 1 day forecast says that it won't happen. Later I could send a notification again, that I was wrong by cancelling it, and it will happen in 30 mins. When the users prepare for an event for hours, then this would be a really bad scenario, because they would miss the event. The other bad scenario that I don't send out a cancellation notification and the event does not happen, while they prepared for it for hours. It would be nice to avoid both somehow. Any ideas?

Answer 1

Your problem falls into the category of problems that involves a decent amount of uncertainty in results. And since you have a broad scope (like weather forecast, stock price prediction, winner prediction in a football game or so) which doesn't fall under a specific business domain, it makes it a lot trickier to provide a prediction/forecast with conviction.

Consider this, prediction or forecasting is generally done based on past events and trends and generally the earlier you try to predict a trend, the more possibility of it deviating from your forecast, since you are providing a greater window for the trend to change. This is why results from Machine Learning models come with a certain probability, and almost always there are conditions that your algorithm has not thought of due to which your results will not be right.

So, what can be done here. Try to define a probability of your prediction happening, along with the prediction. For instance, your weather forecast could have probabilities for different times.

There is a 70% probability of thunderstorms 72 hours from now

Which gives you the uncertainty window of 30% allowing for change, but also letting users know there is possibility of change. This probability will improve when you are forecasting for 3 hours from now.

Likewise with sports, If you consider, let's stay, the trend of the last 10 games and predict that a team would win the 5th match from now, you give it a say 75% probability, but if the team loses 3 of the next 4 matches, that odds will come down.