monitoring
Continuously collecting and watching measurements of a system so problems are noticed quickly.
In more detail
Good monitoring watches latency, traffic, errors and saturation, and alerts on symptoms users would feel. It answers whether something is wrong; observability helps explain why.
Examples
For Kafka: broker health, disk use, under-replicated partitions and consumer lag.A check that fires when memory use stays above 90 per cent is monitoring: someone chose that number in advance.Related terms
- observability — How well you can understand what a system is doing, and why, from the data it emits: mainly logs, metrics and traces.
- metric — A number measured over time, such as requests per second or error count, that shows how a system behaves in aggregate.
- saturation — How full a system’s most constrained resource is, such as its CPU, memory, connections or queue.
- consumer lag — How many records a consumer group has not yet committed on a partition: the log end offset minus the committed offset.
- Kafka cluster — A group of brokers that share the same topics and metadata and act as one Kafka service.
- alert — An automatic notification sent when a monitored condition, such as a high error rate, stays true.
- dashboard — A page of charts showing several metrics together, used to watch a system at a glance.
Taught in
- Load Testing — Modern API Architecture: REST, GraphQL & gRPC
- Observability: Logs, Metrics, Traces — Modern API Architecture: REST, GraphQL & gRPC
- The Alert That Came An Hour Late — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty One Under-Replicated Partitions — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Upgrade That Could Not Start — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Controllers At Once — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Incident Lifecycle — DevOps & Site Reliability Engineering
- Watching Lag And Sizing Partitions — Apache Kafka & Event Streaming
- When Kafka Is The Wrong Tool — Apache Kafka & Event Streaming
- Observability Versus Monitoring — Full-Stack Observability: Metrics, Tracing & Logging
- Why A Histogram Beats An Average — Full-Stack Observability: Metrics, Tracing & Logging
- Prometheus Scraping And The Pull Model — Full-Stack Observability: Metrics, Tracing & Logging
- Off By Five: What Prometheus Is Not For — Prometheus Certified Associate (PCA) Exam Prep
- The Server That Would Not Die: Selecting Data — Prometheus Certified Associate (PCA) Exam Prep
References
- Google SRE book: Monitoring distributed systems
- Apache Kafka: monitoring
- OpenTelemetry: observability primer