I'm designing an API and have reached the topic of logging. I'm going to store my logs in Elasticsearch.
I'm certainly going to do some logging at the time the HTTP response is sent back to the client, with info such as processing time, response code, user id, URL.
Is it best practice to also send a record to the logging system right when the HTTP request enters the API server?
What I have in mind here are situations when the response never materializes, e.g. because the server dies, or takes forever processing the request (e.g. due to bad business logic). If this occurred, I'd have no record at all of the client making a request.
Original wording below (to understand some of the answers). I tried creating a new question with the new wording and then attempted to delete this one. However, SO wouldn't let me delete this, and the new one started getting marked as a duplicate.
API logging: request, response, or both?
I'm designing a REST API and have reached the topic of logging. I'm going to store my logs in Elasticsearch.
Is it best practice to log both HTTP request and response, with some correlation id to match them in the logs? What are the advantages and challenges of doing it this way, as opposed to only logging requests or responses?
(I have some thoughts on this of my own: suspect it is best practice and see some advantages & challenges, but feel there's a lack of an expert treatment of this subject online. Hoping this question will result in one.)
I'm NOT asking about whether to store in the logs the contents of every request and response. I'm asking whether to store some basic record for each request and response (e.g. timestamp, URL, IP, response code, some form of user id), or maybe just for requests, or maybe just for responses.