Log files are a critical part of any serious application: if the logging in the app is any good, then they let you see which key events have happened and when; what errors have occurred; and general application health that goes beyond whatever monitoring has been designed in. It's common to hear about a problem, check the application's built-in diagnostics (pop open its web console or use a diagnostic tool like JMX), and then resort to checking the log files.
If you use a non-text format, then you are immediately faced with a hurdle: how do you read the binary logs? With the log-reading tool, which isn't on your production servers! Or it is, but oh dear, we've added a new field and this is the old reader. Didn't we test this? Yes, but nobody deployed it here. Meanwhile, your screen is starting to light up with users pinging you.
Or perhaps this isn't your app, but you are doing support and you think you know it's this other system, and WTF? the logs are in a binary format? Ok, start reading wiki pages, and where do you start? Now I've copied them across to my local machine, but - they're corrupted? Have I done some kind of non-binary transfer? Or is the log-reading tool messed up?
In short, text-reading tools are cross-platform and ubiquitous, and logs are often long-lived and sometimes need to be read in a hurry. If you invent a binary format, then you are cut off from a whole world of well-understood and easy-to-use tools. Serious loss of functionality just when you need it.
Most logging environments strike a compromise: keep the current logs readable and present, and compress the older ones. That means you get the benefit of the compression - more so, in fact, because a binary format wouldn't shrink the log messages. At the same time, you can use less and grep and so on.
So, what possible benefits might arise from using binary? A small amount of space efficiency - increasingly unimportant. Fewer (or smaller) writes? Well, maybe - actually, the number of writes will relate to the number of disk-commits, so if log-lines are significantly smaller than the disk blocksize, then an SSD would be assigning new blocks over and over anyway. So, binary is an appropriate choice if:
- you are writing huge amounts of structured data
- the logs have to be created particularly quickly
- you are unlikely to need to analyze them under "support conditions"
but this is sounding less like application logging; these are output files or activity records. Putting them in a file is probably only one step away from writing them to a database.
I think there's a general confusion here between "program logs" (as per logging frameworks) vs "records" (as in access logs, login records etc). I suspect the question relates most closely to the latter, and in that case the issue is far less well-defined. It's perfectly acceptable for a message-record or activity log to be in a compact format, especially as it's likely to be well-defined and used for analysis rather than troubleshooting. Tools that do this include
tcpdump and the Unix system monitor
sar. Program logs on the other hand tend to be much more ad hoc.