0

We have a .NET 6 application that consists of a REST API running in Azure App Service, and long-running background jobs hosted in Docker containers. These jobs are triggered by messages in a message queue that is fed by the REST API.

Logs from the web app are written to Application Insights, which works well for that purpose.

But I'm not sure how and where to store logs from the job workers. An important requirement is to be able to fetch the full logs for a job run via REST API after the job is complete.

Application Insights does not seem like a great fit for this purpose for several reasons:

  • Different types of log messages are stored in different tables, e.g. exceptions are written to exceptions, regular log messages to traces. To get the full log for a job run where some exceptions might have occurred (logged and handled properly), we would have to query messages from multiple tables and merge them
  • Application Insights has an ingestion delay, i.e. if the user tries to fetch the logs immediately after the job finishes, the log might be incomplete, and I don't think there is a way to know if all messages for a job have finally arrived
  • There is additional complexity because all log messages from a job would have to contain a correlation ID that is unique for that job

An alternative would be to just store logs for each run as separate files in e.g. a blob storage account. But then we would lose the ability to query logs with KQL, which is beneficial in some debugging situations.

What are the best practices for storing logs like these? Is Application Insights usually used in this situation or is there a better solution?

1 Answer 1

1

App insights is a good fit for most logging.

If you just want to log what happened to your backend jobs, I would add some trace logging for the various step maybe a custom dimension for your correlation id.

You have two stand out requirements though which I think push your senario beyond what is normally considered "logging"

  1. Query the log for a job via an API
  2. Have the log immediately available once the job completes

Given these requirements it's more like the log messages are part of the result of running the process. I would store them along with however you are storing the process result, presumably some database. You can then make the various api endpoint available.

I would still log exceptions etc to app insights along with the logs from your application.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.