1

I have been working on a cloud-based background worker service, and regrettably it is almost "blind". What I mean by this is it is incredibly difficult to get any insight into what is going on, aside from the tedious process of reading through noisy logs.

Some of the common things I would like to know are:

  • is it running?
  • if/when did it crash
  • the result of business logic calculations
  • data used during failed conditional checks

As I mentioned I can pull this information from the logs, but it is noisy from dependencies and other tools logging at the same levels. So I would like to prioritize key performance data from other log details.

One way I could obviously resolve this is by creating a separate custom dashboard UI. This is do-able with the talent on our team, but the time and effort required may not be justifiable.

What I'm looking for are good strategies for getting insight into a background service that are better than looking through raw text logs, while also being less time-consuming to build than a custom UI?

Improve this question
1
  • 1
    What about creating a hybrid solution, a tool to parse the relevant info from the logs and sending a report of some sort?
    – PmanAce
    Nov 18, 2016 at 19:41

1 Answer 1

Reset to default
1

If you already have the log files and they contain all the data you need but are just too hard to go through, you might use the ELK stack to extract the data you want.

The ELK stack is short for:

  • ElasticSearch
  • LogStash
  • Kibana

enter image description here

Installing and configuring the tools might be faster than building your own dashboard.

If you think this setup is too much and just as time consuming as building your own dashboard, then user PmanAce provided you with a good solution in his comment.

Improve this answer
2
  • This will still require a way to categorize entries. Why not make another step and just add a dedicated prefix to service's messages? This would allow using simpler tools.
    – Basilevs
    Nov 20, 2016 at 11:16
  • @Basilevs: I added an answer for "the tedious process of reading through noisy logs" with tools that allow you to draw multiple conclusions by looking at the same data in multiple ways. But I agree with you. Use whatever gets the job done.
    – Bogdan
    Nov 20, 2016 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.