When collecting exceptions, be careful with two things:
What will happen if an exception occurs, and while reporting it, the reporting mechanism throws another exception?
The worst case is to start reporting the new exception, which may trigger a new one, resulting in thousands of new exceptions thrown in a loop. You absolutely need to avoid that.
What if you can't report an exception?
Sending an exception to a server may fail because the server is not responding. Storing an exception to Windows Events may fail because of the lack of permissions. Saving the exception to a local database may fail because the database is, for some reason, down. Even appending the exception to a text file can be problematic: you may simply be out of space or lack the permissions.
Usually, you need to have two or three ways to log the exceptions, doing a fallback from the one which failed to the one which should be more reliable (say report to a server; database; simple text file).
It is essential, once the original problem which prevented the exceptions from being logged properly is solved, to be able to retrieve the exceptions from all those sources and to analyze them carefully. For example, how do you collect the exceptions which were stored in a text file when both the network and the database were down?
When it comes to sensitive data, be sure to encrypt the exceptions when sending them to a third-party server (or even your servers), in order to prevent access to stack trace to unauthorized persons. Make sure that developers never include sensitive information in the messages of the exceptions. For instance,
AccessDeniedException shouldn't have as message:
Failed to authenticate the user John: the password "sihjdfogdhf" is invalid.
Among the information you collect, make sure you have:
The exact version of the application. If you don't have the version, debugging an exception may quickly become a nightmare.
The information about the environment. An application which throws an exception on Windows XP won't necessarily throw it on Windows 8.1. An application which throws an exception with FAT32 system may work well with NTFS. An application may behave strangely because there is less than 1 MB of disk space left, or because the RAM is full and the OS has to swap everything, leading to slow response rates and lots of timeout exceptions.
Something which identifies the customer (if appropriate). This makes it possible to contact the customer to ask for more information.
The most important step is to deduplicate information about the exceptions you collect. Many exceptions will be similar, but not identical: it is crucial to identify how close they are and to group them. The goal is to be able to focus on exceptions which are the most frequent: if you have an exception which affected one user twice for the past year, and an exception which affected thousands of users thousands of times for the same period of time, the first exception may not be your top priority.
You probably won't get the grouping right from scratch; instead, study the exceptions you already gathered, try a grouping approach and see how it's working, then modify it until you get what you need. The grouping is tricky, because there are practically no absolute duplicates, and because the gap between what makes it a duplicate and what doesn't is fuzzy.
If you receive two
AccessDeniedExceptions reported at login window button
OnClick event, one concerning John, another one concerning Mary, those may be grouped, despite the different message strings.
On the other hand, there may be another
AccessDeniedException concerning Mary which happens somewhere else in the application, and which shouldn't be grouped with the first two.
Now there is an
AccessDeniedException thrown by a different revision of your app: the stack trace is different, but the exception is still the same than the first two.
When grouping, make sure you keep the individual information as well. For example, the report may tell that there were 94
AccessDeniedExceptions thrown for the past day for users John, Mary and 14 others in versions 4.0.17 and 4.0.18 with the given stack traces by customers 1, 2, 3 and 4.