The current system I am working on goes through a set of documents & performs some logic using the metadata of the document. If the metadata of a document is fishy..it throws an exception. The system logs the exception, fail the job & let the developer fix the issue by looking at logs before moving on.
I want to improvise this system by making system fault tolerant as I have seen that generally the job fails because of few documents. System continues even if there's an exception raised. The exception information would be stored in the database table along with document information. This information would be sent to developer who can then work on fixing it without having customer waiting. The customer can later process those failed documents separately.
My question is how to store the exception information in table. The exception would already be logged in the log file. One idea I was thinking was to store the time stamp of the exception so that developer can later find the time stamp in log file and understand the exception. But this comes with a possibility that the log files might be deleted.
Does it make sense to store the entire stack trace in the database?