I'm writing a web app which allows users to upload real-estate listings. Part of what this does is allow them to upload photos of the property they're listing.
I'm using a micro-service type approach so there's a property media service which handles all the uploading of media, querying of what media are available for each property, etc.
However because I need fast searching and display of a property page without calling off to 7 or 8 services, I also maintain an Azure DocumentDB collection of properties, where each document holds everything about the specific property needed to support search and binding of the property details page.
So far so good.
I opted for an event sourcing pattern whereby when a new photo was uploaded, I simply raised an event onto an Azure Service Bus topic. There was a subscription to this topic which was a trigger to an Azure Function that would load the document, update it with the uploaded image details, and save it.
It worked - but then I have a parallelism / concurrency problem whereby if a user uploads 10 images, it raises 10 events onto the topic. With the nature of Azure Functions, it might spin up 10 processes concurrently each handing a message.
Because Azure DocumentDb does not support partial document updates, the Azure Function takes these steps:
- Loads the document
- Adds a child to the
- Saves the document
I've recently noticed that I don't get a reliable update when uploading multiple images. Maybe only 6 of the 10 will show in the DocumentDb. The logs of the function show all 10 executions were successful, so I can only assume here that I've been the victim of them running in parallel and one update overwriting another.
To get around it, I've made it so that the media service doesn't raise an event any more, but it just updates the DocumentDb itself. But I'm not really happy with this because it's not really its responsibility in my mind. It's solved the problem for now - but I liked the event-sourcing and Azure Functions updating the document as a better design.
Are there any better options for me? I considered that I could add a document to the DocumentDb collection in the Azure Function instead of updating one "master document" but then it'll mean more work in the property details page to aggregate it together.
Have I missed a trick?