I am working on an application that reads a log file, looks for a specific string, sends alert emails and saves data from the file into a database. The reading functionality is working well, but I am coming across an issue when I stop and restart my program. Basically, it always starts from the beginning of the log file, so I am rereading information I have already read. Please reference the example below.
Consider I have the log file with the following lines
1 0708 1200 Error in log 2 0708 1230 Received invoice 00001 3 0708 1231 Received invoice 00002 4 0708 0130 Error in log 5 0708 0135 Received invoice 00003 6 0708 0200 Received invoice 00004 7 0708 0230 Received invoice 00005 8 0708 0235 Error in log
In this example, say I ran the application. The application would start reading at line 1 and continue until I stop the application (picking up any lines that are added to the file as well). Let's say I stop the application when I get to line 5. If this were the case, I would have received an alert e-mail from line 1 and line 4. I would have also saved lines 2 and 3 in the database.
My problem/question comes from when I want to restart the application again. Currently, the application will start reading at line 1 again. This is not an issue when it comes to saving data because I have placed a clause to not save if the line has already been added. However, when it comes to sending e-mails I will receive an e-mail for lines 1 and 4 again.
After all of the that rambling, I will present my actual question / inquery. Is there a way to stop a program (it completely stops the process) and begin reading from the spot in which I stopped reading from? I cannot think of a way to save the place holder for the file and use it once I start the program again.
I have considered saving the file pointer value to a text file and reading from there at every start up, but I was wondering if there was another way or any suggestions to improve this process. I know this question is abstract, but I am reaching out for extra brains on this issue.
- This is a java program
- This is a web app running on a tomcat server
- The server that tomcat is running on is a unix server
- This is a constantly running application that should only be shut off during an issue or change
- The logs being scanned are very large with 500,000+ lines
- The logs turn around (get replaced with empty log) once they reach a certain size. This means that when I save the file pointer, I have to make sure I am in the same file before I start reading again, otherwise I want to start from the beginning
Any help / suggestions would be very helpful. Sorry for the wordy question, please let me know if anything is unclear. Thanks!