I have a Python based desktop app that can run on Windows, Linux, and Mac. Each time I make an update, I would like to test that it works on each OS, because I have already had several OS-specific bugs.
I know that I could install different operating systems on VMs and login to each VM to run the test suite, each time I make an update. But is there a more automated approach?
Multiple VMs is the way to go, but you should not really have to "login to each VM to run the test suite": you should be able to automate that part.
You can set up all VMs to look at a common shared folder of the host machine, and run a process on each VM which checks the contents of that folder, say, once a second. When a new version of the app is written to that folder by the host machine, each VM should pick it up and run the tests on it.
You could even have each VM store the results of the tests into that same folder and have them displayed by the host machine on one screen. (Or simply have the host machine give a summary, like "all OK" or "win: success; linux: success; mac: failed".)
sshif it is available. – Basile Starynkevitch Apr 15 '15 at 12:55