My application needs to perform an operation once every 10 minutes, but that operation is triggered many times more frequently. Occasionally it is triggered three or four times a second in different processes, and that's the main thing I want to avoid.
It's Django, and at first I used the
django.core.cache feature, but I am still getting more than one worker performing the operation simultaneously.
I have Redis also, and know that WATCH and MULTI can be used to prevent race conditions, but the django-redis library doesn't support them.
This is my solution; please comment on how well (or not) it prevents multiple workers (processes) from performing the operation simultaneously:
- Have a MySQL table with fields
uuidand just a single row starting with both NULLs.
- When a worker is told to start the operation, it quits early if
whenis less than 10 minutes ago.
- Otherwise, generate a UUID and store it in the table, updating
NOW()in the same query.
- Then, select the row and quit early if
uuiddoesn't match the one just generated.
- Otherwise continue with the operation
This relies on MySQL enforcing the order of reads and writes and enforcing that only one query access the row at once, which may be invalid assumptions.
Any alternative ways of keeping Celery processes from running a task simultaneously would be just as helpful.