When should we separate common parts of multiple services into one separated service

Question

We have some services (consider 10, some of them already exist and some are yet to come). All of these services have a common part where they keep track of what's being changed, and these so-called "logs" are shown to users, and the persisting/searching pattern is also common in between all of them (searching might be different in some few parameters). So each service has 2 separate tables, namely log_action and log_details. The first one keeps the higher-level action that happened, and the latter one keeps the details (1-n relationship), consider the following:

User <USERNAME> changed <BIGGER_ENTITY_NAME>'s <FIELD_NAME> on <DATE_TIME>(stored in log_action):
    - Changed the <FIELD_NAME> of item <ACTUAL_ENTITY> from <FROM>$ to <TO>$. (stored in log_details)
    - Changed the <FIELD_NAME> of item <ACTUAL_ENTITY> from <FROM>$ to <TO>$. (stored in log_details)
    ... (stored in log_details)

I have 2 approaches in mind:

1. Creating a dedicated log service (using a NoSQL to achieve flexibility)(with the appropriate SDK for client usages):

   • (+) This approach reduces boilerplate code and tables
   • (+) Takes less time-effort, specially helpful for our new service developments
   • (?) It probably can be considered as a whole service, it manages some resources called "logs"
   • (-) Yet it's keeping some data which is basically owned by other services

2. Let each service manage their own logs (almost all of these services use RDBMS):

   • (+) Actions are private to the service
   • (-) Takes more effort developing new services

Now my question is, what is the best practice here and why? Should we consider a separate logging service? Or let each service do this on their own? Or anything else maybe?

Answer 1

If the shared part is extensive, you could use a Monorepo so that the logging code is maintained only once, but deployed in several different microservices. This avoids adding network logic to what should be simple functionality, and also the overhead of running a node that doesn't contribute to solving the domain tasks.

Answer 2

yes you should, however, you have to ensure that your solution is performant. Logging doesn't lend itself to "normal" REST style apis over HTTP

For example. If you are simply writing whatever you receive to a nosql db, how much "server side" logic do you need? can a shared logging component be 90% a wrapper of the db client?

Similarly if you are writing to syslog or windows events, the client side code is really all already there. You have a second out of process component pick up the various local logs and combine them on your reporting platform.

Performance is everything with logging, because you tend to have a lot of data flowing and most of the time you don't want it to interfere with your applications primary process.