I am looking for a system for building state machines with timed/scheduled transitions, and with events that happen periodically during a given state. I can implement it myself, though I feel like this is a common enough requirement that there would be a category of software for it.

I don't need a library recommendation, vendor, etc. I just don't know what to call this thing, and thus what to google, because it isn't a "cron" and it isn't a "message queue". I don't know if such a server-side layer even exists currently, but I'm hoping to find an indication one way or the other. If this type of question is a reason for down votes and close votes, definitely indicate that, and maybe point me to another SE, or suggest improvements.

The application I have in mind is a timed video game "event" system, like those you might see in an MMORPG (periodic monster spawns, timed escalating waves, failure timer, scheduled event starts), or in a trivia bot (per-question timer, time between questions, round timer), etc. It won't be used for simulations, physics, etc, so I won't be throwing real-time (sub-second) granularity events at it.

The system I have in mind would be some sort of hybrid pub/sub and scheduling system, operating entirely server-side. It would be somewhat like a Javascript setTimeout, except with a more reliable mechanism, probably with a 1 second resolution, and with the ability to pass parameters explicitly when you schedule the timeout. It probably would also have some sort of global monitoring for ops purposes, and maybe would have some sort of support for scaling to multiple systems.

I don't know if I will need to roll my own, or if there are more generic systems that already do what I want them to do. I would prefer to use an existing system if possible. There is some common server tech that is similar, but I don't think it quite does the job.

The system I want operates sort of like Cron, with these exceptions:

  1. It should be function callback/micro-process oriented rather than shell oriented.
  2. Tasks should probably be executed in a delegate sub-process pool, or in-process of the daemon, rather than spawning a new process per task.
  3. It should have a resolution of 1 second (or smaller)

The system I want also operates sort of like a message or task queue, with these exceptions:

  1. It should allow for periodic tasks (already possible in some queues, I think)
  2. It should support parallel execution for tasks that are scheduled for a given tick, rather than sequentially running them off a queue
  3. It shouldn't expect tasks to be long-running, and shouldn't build its interface for long-running tasks
  4. It should be just as easy to make recurring tasks as one-off tasks, without having to build task-chaining glue
  5. I should be able to cancel scheduled tasks easily and quickly
  6. Robustness of preserving and guaranteeing task execution is not a priority over per-task overhead. Vertical and horizontal scalability is way more important than carefully preserving data (at this layer)
  7. Tasks should probably be able to share state at some level (so that IPC and task execution latency/overhead is minimized)
  8. If the system supports horizontal scaling, I should be able to give grouped tasks an affinity so that they are executed off the same shared state, rather than sent to an arbitrary worker in a pool

My preferred programming platform is python, but I would be fine with a stand-alone system in any language that allows for low-latency server side IPC over a standard protocol, so I can use my server-side language of choice. I'd also be fine with a system that specifically integrates with popular server-side languages, as my task implementations are likely to be a pretty small portion of my larger system.

Having a built-in ability to optionally send events to clients in a push style would also be cool, but definitely does not have to be part of this library or layer.

  • 1
    Is this a job for SignalR, or maybe some equivalent tech for your specific stack?
    – RubberDuck
    May 3, 2016 at 8:51
  • @RubberDuck Looking quickly at the SignalR page, it looks like it solves the push notifications, but I'm not sure if it does much to solve the timed workflow part of the job. The timed workflow part is what I'm most interested in solving, and the pipe to RPC back to the client is less important to me. That page mentions service bus, though that looks like a server-side messaging protocol to allow IPC between disconnected services, rather than a task/workflow scheduler. May 3, 2016 at 9:17
  • 1
    Also, let me know if this is not the right place for this, or if my question is written poorly. Just down voting isn't going to help me out here ;) I know way more about SO than I do about this site. May 3, 2016 at 9:35
  • Not sure why it was downvoted. Maybe it's being perceived as a resource request?
    – RubberDuck
    May 3, 2016 at 9:36
  • 1
    sounds like a "scheduling engine" is the term you want. However, the problem with schedulers is what they do when the times comes, and that varies according to the system its written in/for - eg cron works fine for system tasks but won't work like a library for scheduling program tasks.
    – gbjbaanb
    May 4, 2016 at 14:07

2 Answers 2


To be honest, this make me thinks to the JavaScript loop with setInterval and setTimeout and closures though i'm really not in JavaScript.

However here are some problems : if you're tasks need a shared context and can be executed in parallel you may go for some trouble. Events queue aren't mono threaded usually for nothing.

There may be a name for this design, however considering your very specific requirements (functionalities, performance), i don't think that ONE design pattern alone will fit.

The only thing that i see that look like to this is a Workflow Engine. I used Activity, it's a light workflow engine made to be able to process thousands of tasks easily. Tasks have a context which is in the Process. However i don't think this will fit for going below the second.

You were talking about MMO. Do you know you have two different "clocks" : one is the famous Frame Per Seconds that everyone knows, the other is the clock that handle communication with the game, each of them are independent. They basically schedule their own tasks separately because even if they're basically Task in the most Abstract terms, the requirements for them are totally different.

Having in the same queue tasks that must be executed 10 times in a second and other only once per hour with all the other requirements would make your "queue" (or whatever) a god object in my eyes. Really too complex and specific to put a name on it.

So you may need to split your problem and change the design of your tasks accordingly in multiple independent problems :

  • Tasks with shared context
  • Tasks with multiple execution within the second.
  • Independent Tasks with a long interval between execution.

Each of them could use the same interface, say a facade (or a proxy ?) to subscribe in fact to different queue according to the parameters given. Each of the queue would fill only the requirement it was made for

  • one for high performance,
  • one for shared context,
  • parallelism.
  • some mix : parallelism + shared read only-context ?

In the end, i make each components having the less responsibility possible to be able to focus on what is more important for each of them.

  • Thanks for the reply! My intent wasn't to even worry about keeping track of sub-second logic in such a system. I was trying to use an example of a larger event that might be timed in seconds, being able to somewhat intermix with longer-term events that might last minutes or hours. I agree that having separate queues to keep track of the different granularities makes sense. May 4, 2016 at 6:45
  • 1
    I just came upon that software.intel.com/en-us/articles/… . it's a do it yourself thing but talk about Intel® Threading Building Blockstoo. Maybe it won't fit all your need but will give you a good start ?
    – Walfrat
    May 4, 2016 at 6:54

I've recently been designing something similar. What did you end up using/doing?

In my case I'm calling it Event Scheduler although it fundamentally is the same as what you're describing, which could also be called a Task Scheduler (exactly what Celery labels itself).

The only issue that prevents me from using Celery directly is that I need to schedule tasks far into the future and I need to be able to cancel and update them before the trigger time.

My current solution involves a Cron polling a table of scheduled events and triggering the proper tasks/functions although there are open questions regarding the design of these tables to optimize inserts/lookups.

High level description:

EventScheduler module

Events Table      -> sent, archived events just for record-keeping reasons

EventsQueue Table -> unsent, indexed by `trigger_time`, polled by cron every minute

# using a PostgreSql DB
  • No idea. It was 6 years ago. Sorry man. I suspect I froze on this particular project. Judging from it being Python, I think it was a simple server side game with very basic requirements. I am working on an actual multiplayer game now, with a team, and it uses C#. We're basically using a for loop over a list of event structs, with case statement dispatch. Nothing is parallel. Anything that's not practically instant run entirely offline from the server. So the requirements are nothing like my original question here. In C# I think I'd use some mix of coroutines/task parallel library. Oct 19, 2022 at 6:50
  • If it was massive scale web thing, I might consider some sort of ready made event sourcing solution. I also probably would go for a custom set of systems for a given part of the project, and not try to jam everything into the same architecture. Oct 19, 2022 at 6:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.