On hanging Desktop-applications

Question

I am filled with questions pertaining to the situations where a desktop-application hangs, possibly for minutes.

I've got this complicated game-engine written in c++. Several times during development, I've accidentally found an infinite loop during testing and the program causes the entire system to hang, usually for a few minutes. I used to just restart my computer with the physical button, because I didn't realize it was so bad. I'm using Linux (Ubuntu, Debian), is it any better on Windows or Mac?

Am I responsible for damage to a user's installation-environment or the loss of data if they restart their computer while my software is hanging, and their OS becomes unusable? Of-course, I can test the software loads and be quite confident that it doesn't happen, but I don't know if I can be 100% sure.

Is there some clever way overall, to stop a program from hanging and stealing all of the CPU for itself? Like, lowering my game's priority to the kernel?

If the game has a feature for users to create and share maps, which can include writing scripts sent to the engine's interpreter, there would be tons of ways for them to spawn way too many entities and enable their AI that I'm quite sure they could make the program hang. I'm not sure I can enforce every countermeasure against that kind of mess, especially from the game's release. Am I responsible if someone makes a map that damages someone's installation?

I really hope I'm overthinking this. Is there an easy way to protect my program from hanging? Do processes made by visual-studio-code have worse problems with hanging for development-purposes? Is my kernel mysteriously bad at stopping hanging processes?

I don't need help removing infinite-loops from my code, the concern is that due to its complexity I would miss one.

Answer 1

Theoretically, it is the responsibility of the OS to stop any single application from impacting the rest of the system through resource starvation. However, this is a complicated problem and as you have discovered, there are numerous failure cases. The OS should at least be able to minimize the chance of any permanent damage. As such most modern OSes in a general user context work on a mutual respect principle: The application does its best not to consume excessive resources, and the OS does its best to share the resources available among the applications to prevent issues like stalls.

However, the OS can't do anything to prevent intraprocess stalls, where tasks within an process block on logical constructs that make your application feel like its hanging. For example your game loop not delivering new frames. What the OS can do in that situation is provide mechanisms to allow the user to exit the application in that situation. Have you ever tried alt+tab whilst your application was hanging?

So long as you control the development process of the code, and put in reasonable limits on your system (for the custom maps example, put a limit on how many entities can be in a map), and then effectively test the system at those limits, you should be able to avoid these issues. (You may wish to provide ways to override these limits with suitable warnings as more powerful computers become available).

However, if you wish to load external code, that is where it gets into the complex topic of sandboxing. This is a broad and complex topic with many subtleties. It would be best to start from an examination of what your users expect from code not explicitly part of you system. In a lot of cases (game mods), it is entirely at the users responsibility. In other cases such as web browsers, there is an expectation that the browser will provide a significant measure of protection, and as such most modern browsers run in multiple processes allowing more granular use of OS protection mechanisms.

Answer 2

Perhaps a list of possible ways a program can hang itself and/or the system would help:

• CPU overload -- but usually the OS gives other things time slices too, unless you have grabbed real time priority
• memory overload -- swap can make this better or worse (usually worse); mitigation is to tune your OOM killer or restrict memory use with ulimits or cgroups
• Disk overload -- if your app uses so much disk I/O that other applications are stuck waiting; this can be combined with a memory issue to cause thrashing, which doesn't truely lock up your computer but certainly makes it feel that way and may not be practically recoverable
• Input device lockup -- if your application grabs keyboard and mouse and then hangs, there may not be a way to recover without an out of band input (like ssh from another terminal or kernel SAK. Kernel SAK is usually disabled by default for security reasons, and its actions are typically overkill anyway, so this is the worst solution.)
• GPU overload or crash -- it is possible to overload the gpu so that it becomes slow or seemingly unresponsive. Sometimes it is possible to trigger bugs in the gpu driver that causes a display lockup or crash. This may or may not lock up the rest of the system; if it does, there is no recovery except to reset. If it doesn't, the only recovery may be to ssh in and reboot, and even then sometimes a power cycle is required after shutdown.

I'm sure there's others I've missed.

If your application itself is locking up, it may be possible to add a watchdog within the application to either try to clean things up, or maybe in debug mode, abort completely.

Another mitigation for infinite loops within the application is to design to detect them or allow them. To detect them, in a code section that it is possible to have a stuck loop, leave a hook for a watch dog timer or countdown timer and abort the loop when it triggers. To allow them, either treat the loop as an event, and for the next iteration, have it add itself back to the event queue; or add yields within the possible infinite loop to allow it to be postponed or aborted.