With all the github services available nowadays, it has become quite easy to have automated unit tests for e.g. C++ or Python coding projects.

Together with colleagues I have written a basic install script for Arch Linux in Python that I would like to test automatically instead of manually creating a VM, letting the installer run, booting into the installed system and figuring out if everything works. However, I have no idea how I should approach this as the test would basically need to launch a virtual machine and work inside this virtual machine and I don't see how I could do this with Travis for example.


  1. Is it possible with currently available tools to perform at least some automated tests? (e.g. boots successfully)
  2. How do OS developers test their installers? Manual Test Team?
  • 4
    What is stopping you from writing a script that creates a VM (with Docker this could be done on your workstation, no cloud needed!), loads the script, runs the script, and checks for certain conditions that can determine if the script was successful? Commented May 17, 2017 at 19:51
  • I guess my lack of knowledge on how to perform this in a script and not interactively. If we take virtual box as an example (never worked with Docker containers, on my own machine), how do I pass on a command from command line into the virtual machine?
    – NOhs
    Commented May 19, 2017 at 15:33
  • I don't know how to do that with VirtualBox. With Docker it's pretty easy. For example, it could be as simple as this: docker run myContainer:someLabel someCommandToRunInContainer. Commented May 19, 2017 at 15:38
  • Elaborating a little on this and adding some resource references to get one started on such a project would certainly qualify as an answer
    – GPMueller
    Commented May 19, 2017 at 15:55

2 Answers 2


This might not be what you're looking for, but have you considered snapshotting just before you do the install, and restoring to the snapshotted state? Your answer said you don't want to do this with VMs, so apologies if this misses the mark.

I would set up a VM to the pre-installed state, including linking the installation script to a remote share/partition outside of the VM. Take a snapshot, and then do the install. Let the install reboot the instance or do whatever. After inspecting the install, restore to the previous state, fix my script, and repeat except not take a snapshot the second time through.

If you want to automate some graphical repetitive tasks, you could look into SikuliX. You can script it with conditionals based on graphical elements - if you see the "OK" button, click it, etc.

Hope this helps. It's inelegant, but sometimes I have to "cut my losses" and go with the most efficient system.


You could use something like KitchenCI for this.

With KitchenCI you could have Vagrant spin up a bare VM, then run the script you have to install everything that is needed.

Finally, you can have KitchenCI run a test suite written with something like Chef InSpec or Serverspec. InsSpec and Serverspec are testing frameworks that are specifically designed to assert and verify server configuration.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.