I keep reading articles analyzing Monitoring and Observability, or having lots and lots of text regarding how the latter is the extension of the first, or how they are complementary, or how tracing is the next step in APM (Application Performance Monitoring/Management), or any other random opinion on the subject. I honestly can't see anything left after you remove the marketing hype and fluff.
I appreciate monitoring and alerting with a combination of Prometheus and Grafana (or any equivalent stack), and how you can track and visualize metrics collected from application logs, or how the logs themselves can be indexed/labeled for retrospective analysis, and how one can integrate threshold/prediction alerting in that regard.
But at the end of the day, "tracing" sounds like simply adding additional labels (host/container name, service/endpoint, timestamp), and APMs look like a metadata store for service exceptions, RESTful requests/transactions and call stacks, or something high level like that.
Am I a confused dinosaur, or am I missing something? Is there anything more to the above?
OK the above was overly dramatic, but the question remains: How can we do something more than monitoring in a shared application where the only thing we can analyze are application logs? How can we track a transaction through various application levels in a non-distributed, non-containerized deployment (if at all)?