We're having a lot of fun and success using Azure Functions, but looking for advice on two related design issues. Our first challenge is that we need to communicate job state back to the UI. So the customer has a list of items, and each one can have background processing associated with it. The UI should reflect either no state, queued state or currently processing state.
We think the best way to handle this is some kind of state bag (probably in Redis), which can notify subscribers of updates. Then we can use something like SignalR to notify the UI. My concerns are around failures and the state bag not getting updated eventually.
Second challenge is that we don't want a function firing if processing on the object is already in progress, but we don't want to just dequeue it and forget it. This is where functions are a little harder to manage, because if you queue a hundred items for the same object, it doesn't care, and it'll fire off whatever number of instances it can to process down the queue. Basically, we need to serialize by entity. If CustomerID 123 is being processed, we don't want it to be reprocessed until the already executing work on 123 is done. This is where it feels like there's overlap, since we're already thinking about some kind of state bag for the above UI concerns.
Any suggestions for this tool set? We want to process as much in parallel as we can, but serially for like entities.