If story points are about time...
Story points aren't about time. They are about the amount of effort required to complete a task.
The idea of story points is that, regardless of who does the work (their experience, availability etc), the amount of effort needed doesn't change. The time taken may change, but the effort doesn't. So a scrum team aims to determine its velocity over the course of a set of sprints. That velocity is the number of story points it can typically handle. It's an average figure that allows for variations in team composition, absences, tasks proving more complex than thought etc. So it aims to offer a more accurate estimate of what can be achieved each sprint.
If you estimate in terms of time, then you have to factor in the fact that John is less experienced than others, so he'll contribute less and Mary is on holiday for three days so will contribute less etc. This makes calculating how many hours of work can be achieved in a sprint hard. Story points aim to hide all that detail away behind abstract estimates.
The problems with story points are twofold. People find it hard enough to estimate in hours, let alone in story points, so they'll often estimate it in hours then translate that using some arbitrary conversion into story points. That defeats the whole point of story points of course. Secondly, and far more importantly, people will still estimate the story points wrongly more often than they get it right. Estimates are guesses after all. Educated guesses, but still guesses. So the estimate will still be wrong on many occasions.
What is the benefit then? If the team equally estimates all the tasks in MDs, instead of SPs, they will be able to do the exact prediction of what can be delivered by a certain date.
"Exact prediction". Really? In the entire history of software development, I very much doubt there's been a single team anywhere that has performed an exact prediction of how long a non-trivial piece of work will take, other than by pure luck. They played the estimate lottery and guessed the numbers correctly. Estimating is guessing; there is no exact science to it. Story points can help with estimating, but they aren't a silver bullet. The only way to know precisely how long a piece of work will take is to wait until it's finished and then calculate the time taken. But that's not really estimating then...