I'm working on a project that involves the chaining of separate jobs into a single master job, though there may be parallel paths in the chain leading up to the final output. Job and chain details will be stored in a database. Eventually what I want to end up with is a GUI in which blocks representing the individual jobs can be moved around and chained up, with the system definition stored in the DB for execution. This will be implemented as C# on top of SQL Server.

I don't want to reinvent the wheel if I can help it. I'm certain there must be some good patterns out there, particularly for how to represent the flow and dependencies in DB tables, but have been unable to find anything that fits the bill.

More than anything, I'm curious if there are any effective schemas people have used to define the jobs, dependencies, etc. Does anyone know of any commonly used strategies?

  • Have you checked the MS Windows Workflow Foundation? seems that you could easily use that. – Andrew Oct 25 '11 at 14:14
  • I had looked at that, and on the surface it seems very relevant. But it seems to rely heavily on the graphical designer in VS, and it's not clear to me how you could define a workflow in a DB schema, and then programtically construct and execute it. It would be fantastic if someone knew how to do this because, yeah, it seems almost ideal. – Gadzooks34 Oct 25 '11 at 14:34
  • Hi Gadzooks34, tools recommendations are off-topic here, but the rest of your question is on-topic. – user8 Oct 26 '11 at 2:45

I'm not sure if I got your question right, but it seems your are trying to build a tool to graphically specify a directed acyclic graph (DAG) for your atomic jobs, right? That is, a user should be providing dependencies between atomic jobs and your planned tool saves these dependencies to the backend database.

In this case, you could go with the classic DAG schema: One table with each row representing an atomic job, and one table with each row representing an edge in the graph (dependency). Then use a standard DAG visualization algorithm (like DAGmaps) to help in visualization.

  • Exactly. Simply as it might sound, I had trouble figuring out what a rational DAG schema would be. The paper referenced by the DAGmaps link helps a lot. Thanks! – Gadzooks34 Oct 29 '11 at 21:12

Have you by any chanche got a sql server version that supports integration services? That is a great toolboox for organizing a workflow of jobs just as you describe. For referfence please check: http://msdn.microsoft.com/en-us/library/ms141026.aspx

  • That came up in discussion but I don't have access to SSIS on the machines on which this project will be running. – Gadzooks34 Oct 29 '11 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.