"We're logging every http request and response with log4Net."
Although im sure this is very useful for developers, its very bad practice and is potentialy illegal.
As you have pointed out you are probably saving credit card information in a non pci compliant way.
You are probably also processing 'personal information' such as names and addresses etc in ways which the owner has not given you permission to do so. Breaking the data protection act or equivilant law in your country.
You may also be saving your users usernames and passwords in clear text
The solution is simple to say but hard to implement. You have to stop logging every request/response and start controlling the exact message which is written to your logs.
Eg.
Instead of
Log.Write("Error processing request" + request.ToString());
you have Catch the exact error and write
...
DoStuff(publicA,secret1,secret2)
}
catch(Exception ex)
{
Log.Write("Error doing stuff with id : " + publicA);
}
Where you know publicA is some internal db reference or identifier with no meaning to a hacker and can prove it to your security auditor.
To clarify: Neither hiding the logs, having a blacklist of parameters or using a regex to spot and remove secret data really helps you. As you can't be sure that you have caught every case.