Consider a typical imperative design, with mutating variables and numerous references to equally mutating external resources. When something goes wrong with such code, the normal thing to do is to reach for the debugger. The reason for this is simple: this sort of code requires you to mentally step through each stage, whilst trying to remember the current status of all that mutating state. That's hard to do in your head, which is why we rely on the likes of the debugger, print statements and the like.
If this is replaced with functions that behave in a deterministic way, with immutable variables and no external side-effects, then the need to hold a complex state-machine goes away. As you say, it becomes possible to reason what the code does, rather than trying to execute it in your head to work it out.
This is why "functional programming", with its emphasis on immutability, pure, deterministic functions and the like is growing in popularity. The resultant code from this technique is simpler to understand and thus less prone to bugs.
And one doesn't need to use a "functional language" to write deterministic code. I recently wrote an article, "Why declarative programming is often better than imperative, even in C#" that shows how its possible to write declarative, deterministic code in what's seen traditionally as an OO language.