Yes, this is a great approach when fixing bugs in maintenance mode. That way, you at least can demonstrate that the bug was fixed, and end up with a “free” regression test – as long as you have this test, you'll never have to fix the same bug twice.
Writing a suitable test can however get quite difficult when the system wasn't designed for testability, e.g. if dependency injection wasn't used. This doesn't necessarily make tests impossible, it just makes them more expensive and more fragile. In such cases, it might be beneficial to do a very careful refactoring to introduce seams into the design at which you can apply your tests.
In the absence of a complete test suite, good documentation, and good domain knowledge, it is impossible to ensure that your bugfix doesn't break something else. It's good to be aware of this risk, but it can't really be mitigated. Since you are aware of this risk, you will try to confine your bug fixes to the smallest possible area of code. Trying to leave the code better than you've found it is a noble cause, but it's risky if you don't fully understand all implications of this code.
Ideally, you would also run the fixed version alongside the broken version in order to detect changes in the output or behaviour that you didn't intend, but that's probably not feasible for complex systems.
It's also worth pointing out that bugs tend to cluster. If you have the time, ask: why was this bug introduced? What is the root cause? Might it have caused similar problems around here?
With sufficient understanding, it can be better to undertake a more complete refactoring of a module. E.g. one time I was bringing a small parsing function under test. The root cause wasn't that the parsing algorithm was flawed, but the data model of the output. Now that I understood how not to represent this kind of data, I could completely rewrite that part of the code and eliminate a whole class of possible problems. Another time, it turned out that a very faulty module was only present for historical reasons, and wasn't really used any more. Deleting it was far more productive than trying to correct each individual problem.