Do I use Repository or Service Object to Perform Logging?

Question

I'm working in Java Spring, and I have typical service and repository layers. The repository grabs a JSON; passes it along to the service; service maps the repository response to a DTO.

I also need to perform some event logging afterwards (send these events to an auditing REST service), which requires some properties that are not part of the DTO, but rather part of the repository response object. Note that these particular properties are actually used as part of the business logic to perform the mapping.

So, you may say that for obvious reasons, just go ahead and use the repository response object to perform these event loggings since some of properties are not present in the DTO object.

However, mapping of repository response to DTO requires quite a bit of calculations, business rules, etc. which now the event logging also needs if I only use the repository response. In other word, I have to again execute those same business rules which I performed during mapping, and use them for event logging process. All because the event logging needs a few properties that repository response object has but not the DTO object.

To make matters worse, the same DTO object is used by a few controllers to ultimately send results back as part of the JSON.

There are two solutions I pondered:

1. Include those needed properties in the DTO object, so they can be used by the event logging process, but go ahead and mute them during marshalling/demarshalling with the JSON library. If additional event logging scenarios are introduced with more missing properties from the repository response, then I have to keep adding them here
2. Do a portion of logging that requires those missing properties from DTO while performing the mapping in place. The issue is, now I'm heavily coupling the operation of mapping from one object to another with partial or complete event logging which will be terrible for unit testing and general laws of universe. Also, as additional event logging scenarios might come along, this coupling becomes even deeper and deeper

I wanted to know if there are other solutions/design patterns that can be more sensible, extensible, and maintainable that I can utilize?

Answer 1

Given that you want to keep doing your logging where you're doing it, rather then move it to where all this stuff is already known, I'd go with option 1.

What I'd do a little differently isn't to teach something to "mute" them before marshalling. That smacks of knowing more than I want to know. I'd spin up another DTO for logging and send both together in a collection. Then you can hand marshalling what it's always expected and logging can have whatever it's extra little bit of info is without anything but logging knowing whatever that is.

This is an old idea that has been used on errors when you don't want to handle them right away and don't want to branch your process with an exception but you still want the error to get expressed. You just pack up the error and stuff it in the typical dataflow. You can find a great lesson on that here.

Now if you don't like casting when you take things out of a collection I'm afraid you have a little bit of work ahead of you because Java still doesn't offer a heterogeneous collection in its standard library. Joshua Bloch (author of much of Java) has you covered. Either follow that link or pick up a copy of his book: Effective Java.

Answer 2

The short answer

• It makes no sense for a component to need access to a piece of data purely for the purpose of logging it, if the component did not already need that data for its own responsibility.
• The configuration (and therefore shape) of your logs should be handled as much outside of your code as is feasible. At most, the components in your codebase should be deciding the specific log message and what data to pass into it (i.e. whatever is contextually relevant to the future log reader), but nothing else.
• When a lot of your operations tend to get executed in a specific scope, it can be very helpful to create a different/additional logical grouping to your logs (based on said scope) so that the things that are related to the same scope are found closely together.

The longer answer

There are several point on which I'd like to delve into why you've chosen this approach or what your actual end goal is, but this site is not the best for such a Q&A format. Instead, I'm going to take your current situation at face value, though I would suggest re-evaluating your overall approach when you get to these kinds of points where your intended goal clashes which your current setup.

I would strongly advise you to not start adding properties solely for the purpose of logging, because logging should not be driving your core design. Ideally, logging is implemented silently and especially without one layer needing to be aware of another layer's logging needs.

The problem you have can be solved by intelligently grouping your log messages (at the logging configuration level). This way, all of your individual components can log their own things, and the log messages that are thematically connected to one another will be stored close to each other. This solves your current logging need without needing to change your codebase design.

Using a past example of this, I worked in a codebase where pretty much everything that happened was related to a "run". I'm going to sidestep the specifics, but at its very core a "run" was a dynamically created data entity which was used to effectively scope/tenant all other data. Every data operation that a user could trigger, they would trigger for a specific run.

In this project, the end user would always contact us to complain about a specific run. Originally, we separated our logging per layer. There was a persistence log, a business log, a web log. When a support ticket came in, we needed to find the related messages in all of those logs.

I then decided to change our logging approach. Next to grouping all log messages from the same layer; I also started grouping log messages based on the same run ID.
This meant that our log folder contained a specific {runId}.log file which had captured all of the messages pertaining to a specific run. When troubleshooting an issue with a particular run, we could just open that run's log file and its entire history would be there to read.

In our case, I was using NLog, and I effectively logged each message twice: once in the layer-specific log file, and once in the run-specific log file. You may think that this is a waste of disk space, but this was done by intention.

The layer-specific log files would still have their use, e.g. when we were investigating layer-specific issues (query performance, commonly reused features within a layer, ...), but in most other cases we very much needed the run-specific history to figure out what went wrong for a particular ticket.

The added disk space was not a meaningful cost compared to the effort of developers having to trawl through several large log files.

Answer 3

The better is option I found was to create a new object that extends an aggreggate portion of my DTO object (ObjThatIsNeededForBothUIAndLogging ) and placed those extra properties there (ObjWithExtraProps), and use that for my logging purposes. I can then copy properties needed for UI from ObjWithExtraProps to its parent's (ObjThatIsNeededForBothUIAndLogging) minus those extra properties when needed, e.g.:

MainServiceObject (DTO) ---- Inherits ----> ObjectUsedForUI (JSON)
       |           \                                |
  Aggregate         \                            Aggregate
       |             \________ Old Aggregate _____<>|
       /\                                           /\
       \/                                           \/ 
   ObjWithExtraProps ------ Inherits -----> ObjThatIsNeededForBothUIAndLogging  

Answer 4

I'd favor option 1, but with a different approach on how to sort the data. For the sake of example, I am assuming this is your current DTO:

public class MyDto
{
    public string Data1 { get; set; }
    public string Data2 { get; set; }
}

and your event logging needs access to an additional value called string SecretData.

1. Include those needed properties in the DTO object, so they can be used by the event logging process, but go ahead and mute them during marshalling/demarshalling with the JSON library. If additional event logging scenarios are introduced with more missing properties from the repository response, then I have to keep adding them here

From your description, I infer that what you'd end up doing is something like:

public class MyDto
{
    public string Data1 { get; set; }
    public string Data2 { get; set; }
    public string SecretData { get; set; }
}

But what I would suggest you do is to pre-emptively subdivide your data into two subclasses, i.e. the " result" and the "event metadata".

public class MyDto
{
    public Result Result { get; set; }
    public MetaData EventMetaData { get; set; }

    public class Result
    {
        public string Data1 { get; set; }
        public string Data2 { get; set; }
    }

    public class EventMetaData
    {
        public string SecretData { get; set; }
    }
}

Note: I'm using nested classes here but that is not inherently required. Feel free to separately define these classes.
Note 2: Class names can definitely be improved, these are just named for the sake of simplicity.

This keeps things self-contained. Your service receives the MyDto object, uses the EventMetaData for its own internal purposes, and returns the Result object to its own consumer.

This means you don't run the risk of forgetting to hide certain properties from your DTO. Even if the current metadata you need isn't confidential, it eventually might be, and you wouldn't want to risk a leak here.

Also, I really dislike manual labor, and manually removing properties from a JSON response sounds like cumbersome work that's prone to error and forgetfulness.

Answer 5

In the general case you always have the possibility of wanting to see things grouped together in your logs which come from widely separated parts of your codebase.

Rather than refactor your code to allow you to write a single log line with all the information you need, its better to write multiple log lines and then group them in your log viewing tool.

Ie, say you are using Splunk, add a "transactionId" to your logs for everything that happens in a single request, or a userId, or a trackingId or whatever. Then in your query group all the rows together with the transaction command. You can doa similar thing with collapse in elastic or simply searching or ordering by your grouping field.

The benefit of this approach is the separation of your logging and your reporting requirements. You can think up new way you want to report on your logs and not have to change the code.