3

I was wondering how much data should be logged.

I know this deeply depends on multiple factors. But it can still be hard to find the golden middle way.

Lets say I have an application where people can create and administrate a user. Furthermore they are able to create/read/update/delete other objects within the application.

If the application has many users it can be an issue if too much data is logged. If all DTO classes and their contents is logged everytime a user or any other object is edited/delete/etc. the logs could get extremely large. But if you only log a little line saying "User 'blabla' was created/edited/etc." you might have trouble debugging or recreating bugs as you dont have the exact state of the objects when the error happened.

Where I currently work we rarely log data, but instead log error messages and stack traces, but I've also previously worked at places where they logged every single bit of information that can be useful in a future debugging scenario.

I was wondering what other people do and think about this? How much data do you log for future debugging information? - is there such a thing as a "correct" amount of data logged?

4
  • I like the approach used by the semantic logging application block: msdn.microsoft.com/en-us/library/dn440729(v=pandp.60).aspx : have hooks to log everything, but decide at runtime which are active.
    – Patrick
    Commented Dec 2, 2013 at 12:50
  • 3
    you'd think all logging is useless until you need a certain bit of info that didn't get logged Commented Dec 2, 2013 at 12:50
  • I appreciate the feedback, but the downvote is kinda harsh.. I did search before posting, but didnt find any sufficient answer (didnt see that particular post though).. a simple comment would suffice :)
    – Linora
    Commented Dec 2, 2013 at 15:11
  • The approach I take is to log steps and critically associated data with that step. For a single process with 3 classes each with a method to act on a given DTO, I log the fact that the process is progressing into and out of the method, and the data that the method in question is acting on or modifying
    – kolossus
    Commented Dec 2, 2013 at 22:14

2 Answers 2

1

I'm assuming that you are using some type of third-party logging utility that supports multiple logging levels that can be changed without recompiling source code (like Log4J or SLF4J). If you're not, do yourself a favor and implement one... yesterday.

A good logging scheme should allow you to diagnose the root cause of any problem without recompiling code. Obviously, just logging the stack trace isn't going to give you much context. On the other hand, logging every logical step is going to kill application performance and eat up a ton of disk space. The trick is to default your logs to the minimal amount of detail, but allow yourself the ability to turn up the logging level in the event of a problem.

For normal processing the minimal amount of logging necessary is probably best to log at the INFO level. Think of this as a you would a Table of Contents in a book: it's not going to tell you the story, per se, but it will give you an idea on where to look to find what you want. Log your high-level process steps, services, or classes with generic informative messages like "Starting validation process" or "Retrieving query data", but use this message level sparingly (or not at all) in lower-level services, classes, or functions to avoid clutter.

I use DEBUG messages more like breadcrumbs. If the INFO level has done it's job, then I should have a pretty good idea what process broke down. From there, I should be able to turn on DEBUG level and figure out what class the offending action took place. To ensure this, I make sure I log a DEBUG level message at every entry and exit point to every method in every class. On entry, I log the name of the method, the names of the formal parameters, and the values passed in for each of those parameters. On exit I again log the method name, and provide the name and value of each variable returned or state that the method returns nothing.

If even more detail is needed, TRACE level can be turned on for the offending class. Trace level logging should allow you to follow every logical step in the class, almost as if you are stepping through the code using a debugger. Keep in mind that most live applications you will not have the luxury of using a debugger, so the TRACE level is often the closest you can get to that level of feedback. Of course, this is also the most tedious and, in my experience, most often overlooked level of logging. In TRACE logging, try to at least include critical decision points in the method (all paths in conditional statements, pointers in loops, etc) to allow you to follow the path the application took towards its demise.

Back to your example, a DTO class is what I would consider pretty low-level code. You probably don't need any INFO level logging here, but DEBUG on the entry/exit points should be added along with TRACE logging for any key processing or decision making events in the DTO. Then, make sure you turn the logging level on the DTOs to INFO so that these events normally aren't logged.

A couple of other hints: - Default logging to INFO level - Make sure to use a rolling appender to log your output in multiple files rather than one huge file. - Output, at minimum, TIMESTAMP, CLASS, THREAD, LINE NUMBER your log appenders - When adding DEBUG messages logging variable values, keep in mind that the to_string() method of many classes is not overridden so you may not get useful output. Try using the ReflectionToStringBuilder.toString(Object object) static method.

1

I was debugging an app that streamed video data... the logging was insufficient to diagnose a particularly troublesome problem (though the problem was easy once we'd found it), so we had to put logging in the video playback code.. yup, 25 (or so) log calls per second.

So your problem is never going to be "how little logging can I get away with", but "how can I manage truly massive amounts of logging so I can get useful information out of it". The managing of even small amounts of log data is the same exact problem, so concentrate your efforts on that - that means, making sure you can enable sections of logging, or print out parts so they can be clearly identified within a mass of other, useless, log information.

I always log lots of data, and audit it too. Once your code leaves your hands and gets installed on a customer site, and all they can give you to help diagnose a problem is the log files, you can't go back and add more logging to pinpoint the bit that went wrong.

Not the answer you're looking for? Browse other questions tagged or ask your own question.