I want to illustrate my problem statement with a use case. I am building a trading system that acts as a container for deploying trading strategies. The trading strategy comprises dependent tasks. Let's suppose there is task T1 and T2. T1 runs at the market open (the core of the strategy), and T2 runs after the market closes (kind of like an analysis task). The two requirements I have are:
I want T2 to run after T1 and after the market closes. So I can leave system resources for other priority tasks that run during market hours (Note: T1 can complete way before the market closes).
I want the next run of T1 (not the first) to wait till the previous run of T2 to achieve something like a feedback loop.
The way I see it, to accomplish the first requirement, I need to associate the cron information to the tasks along with the dependencies. And to accomplish the second requirement, introduce something like a future dependency (T2 depends on T1, but T1 future depends on T2).
Now, I have used some workflow schedulers like Apache Airflow. But it has cron information at the DAG level, not at the task level. (2.) can be accomplished by disabling parallel runs. Mine will be a single big workflow with each task running according to its crontab schedule.
I wonder if an algorithm/library exists that does the dependency-resolution and cycle-detection for the use case I mentioned above. Looking forward to hearing from Graph(ites)
Please feel free to correct my thinking if I am complicating it :)