I have a set of jobs that I need executed.
Excellent. You have chosen a reasonable architecture to meet this need.
When a job is finished, the job uses a callback function to inform the entity of the result of the job (Whether it threw an exception and such)
Congratulations, you have re-invented
Task<T> and renamed it
Job. Doing the same thing as an established library is evidence that you are on the right track, but it is also evidence that you may be repeating work that is already done.
Maybe just use
Task<T>, if it meets your needs. Note that a task has a "continue with" feature where you can give the callback directly to the task, just as your
Moreover: C# allows you to create task-based asynchronous workflows very easily by using
await. Callback-based asynchrony makes you turn your program logic inside out, and makes exception handling difficult. Let the compiler do the heavy lifting. It will rewrite your workflows into continuation passing style for you, and get the exception handling correct.
If the job failed, jobFinishedCallback in TaskSet will call executeJob again.
That's the wrong thing to do, as you have discovered.
Does anyone have a good suggestion to avoid this problem?
executeJob again. Enqueue the job back onto the work queue again, and return. That way not only do you not get an infinite regression, you also treat the retry fairly. The retry should have to wait its turn; all the jobs in the queue now should have priority over it.
Similarly, when a job finishes normally, don't call its continuation directly; you can run into the same problem. Enqueue a new job which calls the continuation, and give that job a null continuation. And again, this is more fair; the continuation should have to wait its turn, just like every other job in the queue is waiting its turn.
More generally: today would be a good day to do some research on tasks/futures/promises, reactive programming, observable sequences, trampolines, continuation passing style, actor model, and so on. You are re-inventing from scratch technologies that experts have been studying for decades, so learn from their accumulated knowledge, rather than your own trial and error. And use the tested, debugged, well-designed classes in the library, rather than rolling your own -- provided that they meet your needs. If they don't, please give feedback to Microsoft about why they do not.
C# tasks use the concept of "task context" to determine how continuations are scheduled; your re-invention is basically the context rules for GUI applications. The Windows message queue is used as the work queue. There are other choices that are possible; for example, some contexts will instead grab a worker thread off the pool and execute the continuation on the thread. By using the off-the-shelf parts already created you can make use of the flexibility and power already designed into the system.