Best practices for logging user actions in production

Question

I was planning on logging a lot of different stuff in my production environment, things like when a user:

• Logs In, Logs Off
• Change Profile
• Edit Account settings
• Change password ... etc

Is this a good practice to do on a production enviornment? Also what is a good way to log all this. I am currently using the following code block to log to:

public void LogMessageToFile(string msg)
        {

            System.IO.StreamWriter sw = System.IO.File.AppendText(
                GetTempPath() + @"MyLogFile.txt");
            try
            {
                string logLine = System.String.Format(
                    "{0:G}: {1}.", System.DateTime.Now, msg);
                sw.WriteLine(logLine);
            }
            finally
            {
                sw.Close();
            }
        }

Will this be ok for production? My application is very new so im not expecting millions of users right away or anything, looking for the best practices to keeping track of actions on a website or if its even best practice to.

Answer 1

This is not a direct answer to the question, more of an expansion on it.

When you launch a new app I recommend logging everything the user does: log in, log out, scratches their a**, everything. If it's web-based, consider using heat maps so you know what their mouse was doing.

When I was on the BravoX project at Xerox in the late 70's we recorded pixel-by-pixel mouse movements to figure out how users might use this weird thing called a WYSIWYG editor. We would watch playbacks of user sessions during lunch. It was extremely instructive. We discovered a use pattern we called Charlie Browning—the user would select some text and make it italic ... then they would undo ... then they would redo ... back and forth, back and forth. It turns out they were trying to understand this stuff at an emotional level. So we (Greg Kusnik did the code, if memory serves) put in some specific optimizations to support exactly this behavior.

Without the recording we would have never thought to do this.

Answer 2

If I was you and I was sticking with writing to a text file I'd use log4net and log into a specific "UserActions.log" file. This way it doesn't muddle up your normal logging. By using log4net (or any other logging framework) you can avoid re-inventing the wheel and leverage rolling file appenders, warn/error/debug/info codes, batch file writing, etc. It's always a good idea to build in good logging into any production level application.

In reality though, it's probably better to store all this information in a database. Using a database will let you sort it, aggregate it, and do statistics on it easier

Answer 3

Log files are used 1. to get information for debugging system errors. 2. to research user activity for mischief, or 3. to understand how people use the system when you can't go watch them. With that in mind:

• Log environment (e.g. environment vars, other settings, etc.) on application startup. This is helpful for debugging issues.
• Log the user and the action (the distinct part of the URL) for each request.
• Log all parameters for each request EXCEPT for passwords. I like to put delimiters around each parameter in the logs e.g. phone{(999)999-9999} email{[email protected]}. Passwords should never be written anywhere except to the database in a one way, cryptographically secure hash function with unique salt for each user and multiple rounds of hashing (see footnote)
• When logging in, you should log the user's ip address, user ID, name, count of failed logins, maybe the browser, maybe the cookie session id, but never the password.
• Remember to not log the password on the login page AND the change-password page, nor should you log the secret question or answer if you have that functionality. Any other passwords or encryption keys must not be logged. It's a good practice to write something like six stars to the log for these parameters so you can see that you remembered to suppress this data.
• I like to log the total time it takes to serve each request: Done: 49ms
• I like to log changes to session state. These should be rare.
• As others said, there are great logging libraries out there for logging to files, not sure about for logging to databases.
• Store logs securely. Even without passwords, there are state, federal, and international laws about Personally Identifiable Information (see Safe Harbor) making log data confidential.
• Take backups. If you let them flow into a full-drive backup every night, make sure to remember to back them up somewhere else before upgrading to a new server (don't ask me how I learned this).

Other tips

Footnote: To log someone in, hash the supplied password with the same hashing algorithm and the salt from the original hash for that user. If the hash of the supplied password matches the password hash stored in the database, then they are logged in. For this to work, you must define the character set for your password, disallow the Unicode substitution character and any other characters outside your set.

Answer 4

You don't specify whether you are using a database but if you are and that database is SQL Server, then you could add something called AutoAudit and automatically log all interactions with your data. Just make sure to specify only those objects you want audited.

But anyway, I wouldn't be trying to manually code my tracking as it'll end up a maintenance nightmare.

Also, for logging, don't roll your own, use Enterprise Library Logging or Log4Net or similar.

Answer 5

Just a general advice. May not be directly related to your question.

It depends on what are you going to use logs for? Mostly logs are used in the production to detect the operations which cause the errors. If you are keeping them for tracking the user actions, then that's not part of the log. That must be the server side feature of the product. These things then definitely needs to go into database for later study. But the server side logs like, "Error occurred as some text is empty", is not part of the feature. These things needs to go in the file system. They must have following contents with them :- user_id, error_number, error_text, file_name,function_name, thread_id,system_date_time, and any other context.

Now I am only talking about the logs in the files.

1) Keep them asynchronous. I/O operation is costly.

2) Design them as class than function. Future changes will be easy.

3) Keep them singleton, if possible. Singleton is difficult in multi-threaded, so design properly.

4) Also better to keep the interaction between logger and loggee simple. Most of the time send the message_number than actual message_text, and let the logger get the message from the number. This will help if later we want to make changes in the generic log formats.

Normally logger and also the things which we needs to log should be part of the design. I have seen cases where there were changes in design just to make sure that all relevant information is logged properly.

Answer 6

Try Log4Net.It allows you to log in files or database. Here's Tutorial!

We use Log4Net in all of our projects .

Answer 7

Using a log file presents a couple of issues. Firstly you may get errors when multiple processes attempt to access the file. You can also get problems when trying to cycle or clear the file while your system is running. The way around that is to use a database.

So, step 1 is to create a database table. I suggest the following fields:
* userID
* action (eg logon, delete foo)
* Some descriptive text (allow nulls here)
* timestamp

Step 2, Create a stored procedure with input for userID, action and descriptive text. Just use the current time to create the time stamp.

Step 3, Write a logging method in a convenient shared library so it is easy to included everywhere and get into the practice of calling that method as required. You may like to have a logging level flag logic as well to change what is logged.

Step 4, Create a maintenance routine to clear out old messages from the logging table from time to time. Perhaps delete when older than X, run each week or so as part of regular DB maintenance (index rebuilding, etc).

Once you have built this once, you should be able to use the code involved in other projects.