-1

I currently report my production exceptions to a mysql database, where they have been collecting dust for the most part. The problem I faced is grouping the stack traces. I would like to be able to see kind of an aggregate of the stack traces that are the same, so that I can work them into the workflow.

I know one issue I have is I get stack traces in different languages, which throws things off, otherwise I'd just do a group by sql command.

Does anyone know of any libraries that could assist me with this? As long as I have something where I can put in a collection of stack traces and get out the result, I'm fine. I don't need any direct sql interaction, or reporting or anything like that(but I'll take any suggestions you guys have)

Thanks

7
  • How do you mean "the same"? Exactly the same? Same line numbers? Same function calls? Same chunk (between the error reporting and the base system)? Some other "the same"? By "different languages" do you mean "function names in french or english" or "java stack trace vs c# stack trace"?
    – user40980
    Feb 12, 2013 at 1:29
  • Different languages, like human languages. It's the same concept you find if you develop apps for the android. In the developer center(or whatever it's called) they have a list of stack traces, and they are grouped together, with a count of how many happened, if there are any new ones etc. imgur.com/fhSbNv3 Feb 12, 2013 at 1:36
  • @gnat Well if this isn't a good place, then I don't know where is. I know this would have gotten shut down on stackoverflow and it surely doesn't fit on super user or server fault. Tried to post on HelpMeAndOthersFindALibrary.stackexchange.com but apparently it was off topic there as well. Feb 12, 2013 at 16:16
  • 1
    Consider the 'not constructive' - the "help me find a library" can have multiple possible answers that are equally right (the polling part). I approached this as "there is a problem grouping and identifying a stack trace" rather than the "are there libraries" and addressed it that way. You might consider rewriting the question to identify the specific problem you are having - one of the answers may be "off the shelf library" or it may be "roll your own, here's how to do it."
    – user40980
    Feb 12, 2013 at 17:31
  • That's true, I just assumed this was the place to ask those sorts of questions(erroneously). Anyways thanks for explaining it. Feb 12, 2013 at 17:35

1 Answer 1

2

Prepackaged library? Likely not. There exists some - but it doesn't appear to be at the size of where we are intrested in - STAT and a YouTube video on it. Interesting, but designed for systems that are an entirely different scale than most people work on.

In previous searches for similar solutions, ultimately I ended up writing my own stack trace fingerprinting software.

You've got a stack trace from the system and have stuck it some where. It looks roughly like:

com.foo.bar.UserNotFoundException: User not found
    at com.foo.bar.ErrorHandler.handle(errorHandler.lang:42)
    at com.foo.bar.StuffDoer.doStuffA(StuffDoer.lang:314)
    at com.foo.bar.StuffDoer.doStuffB(StuffDoer.lang:271)
    at com.not.yours.base.Base(Base.lang:2357)

This the general format of a stack trace - it has the exception handler at the top (which isn't too useful) and the base of the system at the bottom (which isn't that useful either).

As an application programmer, what you really want is the stuff in the middle. This may be one method, or a dozen, but its what actually identifies this stack trace vs other ones. The specifics of the stuff outside your call isn't always that useful.

As a systems programmer, the stuff at the bottom is likely the most interesting.

The first step is to parse through the stack trace line by line identifying the relevant information and further identifying if the given frame is of interest at all - if everything goes through ErrorHandler.handle() before the stack is reported, that frame is not useful in identifying or grouping the stack.

Next, I serialized the array of the interesting bits of information (method name, class name - line number in the case that I dealt with in the past was too variable between releases that it wasn't useful, but it may be for you) and stored the hashcode of that in a database. Because of the different systems and languages working on the application to identify a stack trace, this was a simple text join and md5sum (if I had to do it again, I'd be tempted to make it into JSON and hash code that).

In the database, the hash code of the interesting part of the stack trace could then be computed quickly and was a fixed size (nice for database indexes and or keys - though not bad for directories either).

Stack traces longer than a certain minimum size were examined multiple times. For example, if the minimum size for further examination was 3, and the stack trace was "a b c d e", the following stack traces would be stored:

  • a b c d e
  • a b c d
  • b c d e
  • a b c
  • b c d
  • c d e

Yes, sometimes everything was a subset of another set of analysis, but it let people look at smaller parts and identify patterns of misbehavior.

3
  • I was hoping to not have to roll my own, either way, thanks for the tips. Feb 12, 2013 at 2:17
  • @kelton52 I've looked in the past, I looked again. In the past it was determined after a bit that anything we did find would be too general to not fill the requirements of the project (and we'd then be welding our functionality onto third party software) or too specific in another direction (such as the STAT tool mentioned) that it wouldn't be useful.
    – user40980
    Feb 12, 2013 at 3:34
  • I'll give it to ya since this has been closed. Feb 12, 2013 at 16:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.