While making an application I've come to the point where I want to add logging for the inevitable case when something goes wrong. Now this seems like a problem that should have been solved decades ago, and indeed - there's no shortage of logging libraries for any decent programming language. But while trying to choose among them I ran across a rather fundamental dilemma.
Do I emphasize safety or performance?
Let's look at both extremes. If I emphasize safety, then I should make sure that the log entry is safely stored before I proceed with work. Otherwise I could end up in a situation where my program thinks it's logged 5 log entries and has done substantial work, but then an error makes these entries disappear and later forensics turn up nonsensical results. "I see that the program only logged up to point A, but there is already data stored which suggests it reached point B. Then where are the log entries between A and B?! What's going on here?!" At the furthest extremes of safety this means that after I produce a log entry I also need to wait until it has been successfully flushed to physical storage. However that's a VERY expensive operation and would basically kill my program's performance, since every log call would probably take tens if not hundreds of milliseconds.
On the other extreme - performance. In this case I need to shove the log entry to wherever as fast as I can and move on without looking back. Another background process then can take said entry and try to write it to physical storage, but the main process is unaffected. The main risk here is the one mentioned before - it's easy to lose log entries if everything crashes and burns before they could be written out.
And this whole thing is on a spectrum, of course. By employing various strategies you can make your chosen approach either safer (but slower) or faster (but riskier). And... I can't decide how to choose.
Is there some commonly accepted middle ground? Has anyone every come up with a way of deciding where your particular situation should lie?