2

We have several application servers (and different applications) running on Amazon AWS and our main method of logging is a combination of statsd, carbon/graphite, and an alert system based off of the metrics coming into carbon. This has been working great for a lot of different situations but I feel we need something else to complement this type of logging. Something that is not as much for streaming or visualization like graphite, but allows easy lookup by time/date, error type, etc... What are some techniques that make reviewing your logs quick and effective? My initial thought is to UDP all logging messages to a central server and insert them into a database (likely MongoDB). Is this a reasonable approach?

1
  • This problem has already been solved with logstash and elasticsearch.
    – user7146
    Commented Feb 24, 2014 at 8:22

1 Answer 1

3

I think you're on the right track. You could centralize logging to a central server and then have logstash consume the logs and store everything in elastic search. Then, you can use Kibana to search / lookup, etc. It's a good complement to statsD+graphite.

http://logstash.net

http://www.elasticsearch.org

http://www.elasticsearch.org/overview/kibana/

1
  • Ribald, I'm in the middle of setting this up now. It looks super promising, thanks for the tip!
    – tonyl7126
    Commented Feb 24, 2014 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.