6

I'm writing an app which uses a logger with different logging levels (info, debug, warning, error, etc.); but - I'm used to the idiom of writing program output to stdout and error information to stderr.

Suppose my logger uses all of these levels, and that I use the logger to report all errors. How do I combine the stdout/stderr idiom with logger levels? That is, should I have the logger...

  • only write to the standard error stream?
  • only write to stdout?
  • choose between stdout and stderr depending on the levels (e.g. info and weaker to stdout, warning and more severe to stderr)?
  • do something else?

Note: It's a C++ app in case you believe that matters.

7
  • 6
    Been here over 8 years and I still don't understand why a good on-topic question gets a reception like this without a single comment. Commented Jun 24, 2022 at 20:57
  • @candied_orange: Also, someone suggested the question be closed because it lacks focus - but did not bother to comment on how they believe I should focus it. This reminds me of what I'd written as an answer to a meta.SO question about encouraging downvotes.
    – einpoklum
    Commented Jun 24, 2022 at 21:05
  • Agreed, @candied_orange. A question like this should be the bread and butter of a site concerned with conceptual questions about software engineering. It did not deserve the -2 score it had. Commented Jun 26, 2022 at 18:02
  • @GregBurghardt: Especially compared to the +16 for "What are the worst things that inexperienced developers forget to think about?"...
    – einpoklum
    Commented Jun 26, 2022 at 19:55
  • 1
    @einpoklum: that question was from 2011, where the scope of the site was much broader (and the name was Programmers.SE, not Softwarengineering.SE). Even for 2011, it was much too broad and a poll, hence closed quickly. Today, such a question would also be downvoted and deleted immediately. Old questions from 2016 or earlier don't give good examples for what the community here accepts today.
    – Doc Brown
    Commented Jul 25, 2022 at 12:51

4 Answers 4

7

For a background process that could be considered a service or a daemon, syslog is appropriate.

For a user program, especially one designed for user interaction, syslog is probably not appropriate, so stderr/stdout is more appropriate.

Traditionally, unix programs are designed with pipes in mind, where you might be taking input from stdin and processing it and sending the results to stdout. The point of stderr is so that messages designated for the user have somewhere to go other than contaminating the output stream.

From that viewpoint, it would be wrong to send any user designated messages to stdout.

However, if your program is more interactive, where it would not make sense to save the output or pipe it to another program, it is probably moot if the messages go to stdout or stderr, and it may make more sense to send everything to stdout. Also, stdout and stderr may be buffered differently, so if you are sending data to both, the errors may end up misplaced within the mixed data stream.

The key question to ask is if it makes sense to separate the program output from the error stream in any situation, or if embedding those log messages in the output stream to annotate it is more appropriate.

3
  • 1
    On topic of buffering - due to its "interactive" nature, stderr is usually unbuffered. This may be a major deal, if you write to stdout, which may be unbuffered in terminal, but may become buffered when redirected to a file.
    – Basilevs
    Commented Jun 7 at 19:28
  • (sorry for the late comment) - the decision of whether the process runs in the background or not is often external to the process itself. It can either be daemonized, or not. Which would raise another question: With stdout/stderr, the daemonizing invoker process can redirect the streams into whichever logs it likes. But it can't control the 6 or 7 different log level outputs which the child process has internally, to make choices about what happens with entries on those levels.
    – einpoklum
    Commented Jun 9 at 13:26
  • This is why many daemons have options to log to stdout, put in background or keep in foreground for debugging, or log to syslog.
    – user10489
    Commented Jun 9 at 20:52
0

choose between stdout and stderr depending on the levels (e.g. info and weaker to stdout, warning and more severe to stderr)?

This. Unless there's a good reason not to I like stderr to get errors. This is customary expected behavior.

When making this call simply pretend that the level logging system doesn't exist. Send the log message where you would have sent it before. Nothing in a level logging system justifies changing the meaning or use of stderr. If you use it, use it right.

A case can be made that warnings don't belong in stderr. A warning is a potential problem but doesn't indicate that anything the system was asked to do hasn't been done. Understand that some sys admins monitor stderr for any text at all and configure automated responses to it.


The stdout and stderr custom is to directly hard code those destinations / concepts into your app and configure where they go on the command line when the app is run (be that, console, file, etc).

The logger custom is to directly hard code logging levels into your app and configure where they go, and if they go, in a logging configuration file that is aware not only of the log level but what logger was used to send them. This configuration also controls where they go (stdout, stderr, file, etc).

They can be configured to work together just fine. The difference between them is that configuring stdout and stderr output at the command line is a 30 second google search. Configuring most logging system with logging levels involves planning. Even if all you want to do is make it work like the old school stdout and stderr.

The reason planning is needed is the flexibility. Log level tools let you make noisy things shut up when you don't need them and bring them back when you do. It also lets you chose which things to do this to. All without touching code.

With stdout and stderr all you get is two buckets and a choice of where to dump them.

So you combine them by configuring the logging tool to wisely target the most appropriate bucket (stdout or stderr) for the log message. Not that you must use either std bucket. But if you use them then people who know how to use the std buckets, and don't feel the need to mess with logging levels, will appreciate not having to touch that confusing config file.

9
  • The only part of your post which actually answers my question is the last paragraph. But - you don't actually say what you believe is a "wise" configuration...
    – einpoklum
    Commented Jun 24, 2022 at 20:00
  • I believe a wise configuration is one that minimizes how much people have to edit the configuration file when debugging. That means using the old two bucket system as fully and wisely as you can given it's limitations. If you're asking for more then please understand, I don't know your app or your requirements. I just know these two systems are not at all exclusive. You can fully use both at the same time. Commented Jun 24, 2022 at 20:02
  • I didn't say my app has configuration files; so that consideration is moot. I intentionally did not say what the application, does, since I'm looking for a more general answer. But I have edited the question to clarify. I'm interested in what you believe/know to be the common, or appropriate, choice here for a generic application.
    – einpoklum
    Commented Jun 24, 2022 at 20:13
  • @einpoklum better now? Commented Jun 24, 2022 at 20:20
  • "Unless there's a good reason not to I like stderr to get errors." <- But not warnings? Or debug/trace info? And - is that just a personal preference, or are you following a custom you've noticed elsewhere?
    – einpoklum
    Commented Jun 24, 2022 at 20:23
-1

In unix systems with syslogd, the logging levels (info, debug, warning, error) are handled by the logging daemon. Your code would just log everything (using syslog calls instead of stderr), and tag each message with one of the log levels, and syslog itself would decide what to do with it.

This would allow for complex filtering of logs separate from your program, and log levels could be changed on the fly without interacting with your program. Generally, however, few people bother to configure syslogd and just leave the defaults (which would be to throw away everything but error). But if someone was trying to debug a problem, this might still be used.

These days, with systemd, it's trivial to just dump to stderr and let systemd take care of it, but this lacks some of the flexibility of syslogd which is still in use.

4
  • But my application is not "system software". It's just something a single user runs. It shouldn't have write access to system logs.
    – einpoklum
    Commented Jun 25, 2022 at 6:54
  • That's a fair point...user software should give errors to the user. But if it is going to run in the background without user input and long term, system logs may be appropriate. And there is no restriction on what can use syslog.
    – user10489
    Commented Jun 25, 2022 at 12:26
  • 1
    That's really not what I asked about. On an unrelated note: systemd is an engineering tower-of-babel and should be discarded.
    – einpoklum
    Commented Jun 25, 2022 at 18:38
  • Noticed it wasn't what you asked, so re-answered, but this answer applies to similar things. Systemd is both better than what it replaced and worse. I don't think it should be discarded, but I can't say I love it either.
    – user10489
    Commented Jun 26, 2022 at 1:58
-1

What I do in my output functions may be useful here. In my application, I check to see if it is running interactively or not. If so, all of my output goes to STDOUT and/or STDERR. If not, all the output goes to stdOUTfile and/or stdERRfile. I have an additional option that allows and interactive session to write to the screen and the file. Here is some pseudocode that I use: (I'm on Red Hat)

; part 1 - interactive?
myTerminal = systemcall (pts -p $$ -o tty=)
if <myTerminal> == "?" then
  ; This is a non-interactive session
  is_interactive = .False.
else
  ; Interactive Session
  is_interactive = .True.
  and_log = .True. or .False.
endif

...
; part 2 - set log files
stdOUTfile = <path/file1>.out.log
stdERRfile = <path/file2>.err.log
...
; part 3 the output function(s)
write_stdout (
if <is_interactive> then
  print &0
  if <and_log> then
    print &0 >> stdOUTfile
  endif
endif
; STDERR would be similar

YMMV

To be fair, I hate clogging up the console (or system mail for crons) with useless stuff. Now, legitimate issues always go to STDOUT / STDERR. Things like Error: Detected insufficient memory to run the application

Now, about how to go about it with logging levels: Critical messages should always go to the user and the log.

The system logger already has a lot of functionality that you can use.

But my application is not "system software". It's just something a single user runs. It shouldn't have write access to system logs. – @einpoklum

Logger has a -f option that allows you to specify which file you want to write to, instead of syslogs. You'll have to check your OS to see how it behaves: it may or may not also write to syslog and it may or may not write to STDERR also. The added benefit is, it can also accept STDIN as a message. Syslog will then make it easy to "tag" your output with error levels to make winnowing down the messages easier.

1
  • Do not write to files if you can avoid it. It is much harder to implement log rotation, systemd integration and really, any integration when data goes to random files.
    – Basilevs
    Commented Jun 7 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.