Are scheduled job servers the right choice for a time sensitive game engine?

Question

I am currently architecting and designing an exciting new web application that will be entering into some areas that I have very little experience in, game development.

The application is not necessarily a game, but there are some very time sensitive tasks and scheduled jobs that a server will need to run to perform game related activities (Eg. New match up starts at noon every day for a 12 day tournament, updating scoreboards at 5pm every day, etc...)

In the past I have typically used cron jobs with the Quartz Scheduler running within a web application server, but I know that this isn't likely a scalable solution for the truly massive userbase that management is telling me to expect (Granted they are management and are probably highly optimistic about this) and also for how important the role of these tasks are in this web application.

The other important thing I want to consider is that I want to avoid SPOF (Single Point Of Failure). If the primary job server goes down, another job server should be able to successfully run the job in its place. I suppose this can be done appropriately record locking and database transactions.

My question is if scheduled jobs like CRON running on a web application server are a wise design choice given the time sensitive game tasks of this application, or is there something more appropriate for running a scalable game engine parallel to the web application servers?

Answer 1

I don't know why you are discarding Quartz as a possible option. Quartz has a clustered deployment configuration that gets you a cluster-singleton redundant scheduler (One node of the cluster will run the job with certain warranties). It uses the db lock technique you mention to achieve cluster-wide locking. See Quartz cluster configuration.

What is not clear from your 'load' requirements is whether a single node in the cluster should be responsible of running the scheduled task (cluster singleton) or whether every node in the cluster should execute said task at a given point in time. The former can be achieved with the clustering options I mentioned. The latter is possible by using independent Quartz instances on every node, running the same configuration and NTP to ensure time-sync across nodes. (There are other options, like sending a trigger message to clustered topic).

The only point where a clustered Quartz with a SQL store does not scale well is when you want to schedule sub-second jobs sequentially, but from your description, that's not the case.