0

I have scheduling requirements where I place all my schedules in SQL db and then use the azure function every minute to read that database. When the schedule is met, I then perform a durable function for that schedule.

This causes a lot of Azure Functions to be invoked, and CPU usage also increases as this is continuously polling.

I am thinking about using Service Bus to replace this but creating a service bus message with delays.

But how reliable is Service Bus? Does it ever fail?

If a Service bus trigger was to fail for any reason, is that message deleted or would that message cause another trigger?

2
  • please don't cross-post: stackoverflow.com/questions/74881746/… "Cross-posting is frowned upon as it leads to fragmented answers splattered all over the network..."
    – gnat
    Commented Dec 22, 2022 at 11:18
  • Yes, i agree, I was introduced to this site, by another user, who suggested this would be better place to put my questions for which I have now done. Commented Dec 22, 2022 at 14:05

2 Answers 2

1

TL;DR

Azure Service Bus is very reliable and like any other cloud resource can have failures, but it has resiliency features like client side retry logic, receive-mode, locks, 'dead-letter' queues, and so on.

For your use case, you can ingest a message to a Service Bus queue few seconds or milliseconds before that "scheduled task" needs to be executed. The Service Bus Queue trigger function will then pick up that message can do the work, or can trigger another Durable function. It uses the peek-lock receive mode. If the function successfully completes, then ASB will remove it from the queue, and it will not be visible to other workers.

If the function failed and returned error code, then that message will still be in the queue - it will be tried to be processed, upto the configured max attempts. If all the attempts fail, then that message will be moved to DLQ.

There is more to know about ASB like auto-renewal of message lock, etc, but in short, you could use ASB queues to meet your requirement described.

3
  • Great, as at the moment, I am using an Azure Function, which triggers every minute to check if any operation time has been reached, e.g. Every Friday at 10 am. But if Function App was down at 10 am for any reason, then that operation would not take place. So would I be correct in understanding with ASB this message would remain in the buffer till everything restarts again, even though it would be past the delay duration? In regards to Max attempts, what is the duration between each attempt? Commented Dec 22, 2022 at 14:36
  • Yes, messages that are not yet marked as consumed, will get processed immaterial of that time passed or not. When you create the queue, you can configure that max attempt, and lock duration. However, the duration between each attempt is not exposed as a configuration parameter though. Commented Dec 22, 2022 at 14:41
  • Thanks So when I send a message, I will set the lock duration. Once a function is triggered, can I set another service bus message with a delay within the same function? For example, I might want to check the status of a device. If the status is not what I require, I might need to recheck again after 30 seconds. So I would need to come back to that function. Commented Dec 22, 2022 at 21:29
0

Everything can fail. Even your db can fail. Normally that is why we use transactions to ensure the db is internally consistent. The problem is you can't have a transaction across your db and the service bus. Meaning if you save to your db first and your service bus is unavailble you have a problem. If you save to your service bus first and your db is unavailble you have a problem.

The best solution is to have one to be the source of truth (your db) and sync to the other. This is what you are currently doing, also known as the outbox pattern.

You can optimize it a bit where you send a message without much content but only "now would be a good time to look in the db" this way you can reduce the polling but if the message were to get lost nothing everything is good

3
  • Maybe as a backup, I can have a process that triggers every 5 minutes, which checks the SQL DB and what should be in Service Bus. Under what circumstances would a service bus not be available? I am also trying to avoid using durable functions in the polling method, as it seems that this increases CPU usage, which I need to reduce. Commented Dec 22, 2022 at 14:36
  • It all depends on the price of failure. If the user can simply try again it may not be very important to spend time on this. If you are losing a transaction on the books are no longer balanced it is a must to handle this Commented Dec 22, 2022 at 15:34
  • We have IoT Devices. Which would be instructed to switch on for example at 10am, but then need to be also instructed to switch off at maybe after e.g. 10-15mins. There could be a possibility after being instructed to switch on, if the service bus was to fail, then it would not complete the process by its switch off command. Maybe I could added another service bus message(At the time of switching on a device), which simply check that process is completed if not then re creates the switch off message. Commented Dec 22, 2022 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.