3

I am attempting to create a long running job to be used in Azure. I would like to be able to run multiple instances of this job if I needed it.
I was thinking of seperating the primary functionality into two different webjobs. With one job called "the brain". while the other job(s) would be called the worker.

Brain would be in charge of actually knowing what jobs needs to get run and put them into the Azure storage ques that would be listened to by worker jobs. I envision a few different queues and a few different workers handling different tasks listening to queues.

The job list is stored in a regular database with values about when it should be run next and the type of job it is. Brain will periodically look to the database and get the next job that needs to get run. Pass it off to a queue for a worker webjob to handle and either write back to the database if the job is regularly run or just update the status in the database.

Question: Given this setup how would you actually handle getting the next job to be run (what is next is based on a date and time) and setting it's status to 'processing' (so it is not picked twice) in such a way that there is not chance for it to be picked up twice (two instances of Brain running)? I am using EF6 in this case. Is there a way to get the record and update it before anyone else could do that on the database?

A shorter version of the question is How can I pull and update a record and be sure it is not modified while the process is happening using EF6?

Should I be using a different type of job management? Is there a good tutorial available on a better way to do this?

  • Have you looked at using something like the Quartz scheduler? It's pretty great for periodic jobs. – nicV Sep 1 '15 at 18:26
  • When we were just doing proof of concept we used a cron job like quartz scheduler. The "Brain" webjob should be better for our production app. Especially since it will be a core process. – Dan Sep 1 '15 at 18:57
  • What type of DB is it? SQL server has queues which should prevent 2 Brains picking the same job. I'd still suggest something like a scheduler, the Brain could just schedule which ever job instead of passing it to another queue to be picked up by another job and then run. – nicV Sep 2 '15 at 7:47
0

A shorter version of the question is How can I pull and update a record and be sure it is not modified while the process is happening using EF6?

I usually use the following. Suppose you give a name, or ID, to each "Brain". You start a transaction, read if a job is already taken ("owner" column = NULL), write your own id in the "owner" column. You commit the transaction, then you read it back. Now the column can be

  • NULL
  • your ID
  • another ID

If it is NULL, you try again (maybe another Brain took it, failed, and you reset the owner to NULL in its finally block?), if it is your ID you "won" and you can proceed, if it is another, you let it process the entry and you look for another one or go to sleep. This is quite flexible; it usually works with plain SQL (and with ORMs on top of it, like EF6) but also with other systems, like redis.

Should I be using a different type of job management? Is there a good tutorial available on a better way to do this?

A different "way" which is usually used with massive distributed system is to let two workers process the same item; if it happens, and the process is idempotent (processing it once or twice does not change the final result), you just let it happen, on the basis that synchronization is more expensive than the actual processing. This model works well with Azure queues. See for example this article

In general, a very good guide for Azure is on MSDN. I also found very interesting this article. It is old, but it is very clear and helped me in the past.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.