We have server that runs data processing tasks, using Python, Django and Celery, on Linux. Once a batch of data comes in, a number of Celery tasks are scheduled and they run, some of them in parallel, some depend on the results of earlier tasks, this all works.
Now we have to implement a new task -- a proprietary Windows-only program needs to be called on some of our data files, resulting in some more files. Only one instance of this program can run at the same time (basically we have to rewrite its config.ini before we can start it each time, and there is only one such .ini). It must run on Windows, can't run under Wine.
Sharing the files is easy, there is a network mount that both sides can talk to.
But I'm not sure what the best way is to start the task from the Linux server, and know that it has finished, that would work nicely with the way we currently do the other tasks.
One option might be to give the Windows machine a tiny webserver, let it listen to a request so it can start up once it receives something, have its use its own Celery or similar, and let it call another URL on the Linux task server once it has finished.
But that seems a bit clunky, do I have better options?