Full-Stack Observability: Metrics, Tracing & Logging

OpenTelemetry, Prometheus, Grafana & Distributed Trace Context

6 modules, 28 lessons: The Foundations Of Observability; Metrics And Prometheus; PromQL And Dashboards; OpenTelemetry And Distributed Tracing; Alerting And Reliability; Incident Response And Postmortems.

Advanced · 2 hours · 28 lessons · 59 labs

Who it is for

Advanced level.

For engineers, on-call staff and security operations analysts who need to find what is wrong from logs, metrics and traces, and alert on what customers feel.

What you don't need yet:

What helps:

Before you start:

Take first:

Take next:

Learning paths:

Nobody needs to know everything before they start. Each course builds on the one before, so a learning path takes you there step by step.

Labs

59 labs, 47 of them graded: you write the code, it runs, and graded labs check the result. The outline lists each lesson's labs.

What is free

Start free: First module free with an account. Pro: Every course, lab, handbook, practice exam and certificate.

Course outline

The Foundations Of Observability

  1. Observability Versus Monitoring (2 labs): You can say what monitoring is good at, what observability adds, and why a green health check is not evidence that a service is working.
  2. The Three Signals: Logs, Metrics And Traces (3 labs): You can say what a log, a metric and a trace each answer well, what each costs, and which one to reach for when something is wrong.
  3. Structured Logging And Why Text Fails (3 labs): You can write log events as records with stable field names, explain why a sentence is not a log, and query your own logs without a regular expression.
  4. Log Levels Done Properly (4 labs): You can choose the right level for an event, set a threshold at run time rather than at build time, and stop using error for things that are not errors.

Metrics And Prometheus

  1. Metric Types: Counters And Gauges (4 labs): You can choose between a counter and a gauge for any measurement, name them the way Prometheus expects, and explain why a counter is only useful once you take its rate.
  2. Metric Types: Summaries And Histograms (3 labs): You can explain how a histogram stores a distribution in counters, why a summary cannot be aggregated across instances, and how to choose bucket boundaries.
  3. Why A Histogram Beats An Average (2 labs): You can show why an average hides the experience of your slowest users, read a percentile correctly, and know what a percentile drawn from buckets can and cannot tell you.
  4. Cardinality And How It Bankrupts You (2 labs): You can count the series a metric will create before you ship it, recognise the labels that must never exist, and say where high cardinality data belongs instead.
  5. Prometheus Scraping And The Pull Model (2 labs): You can describe what a scrape is, configure a target, read the health of every target, and explain what the pull model gives you that pushing does not.
  6. The Prometheus Exposition Format (3 labs): You can read and write the text format a scrape returns, including help and type lines, label syntax, histogram buckets, and the rules that make a page valid.

PromQL And Dashboards

  1. PromQL Basics And Selectors (1 lab): You can select a Prometheus series with labels, read an instant vector, and explain why a query result is a set of time series rather than a single number.
  2. Rates And Increases Over Time (2 labs): You can turn a monotonically increasing counter into a request rate, account for resets, and choose between rate and increase for a real question.
  3. Calculating Percentiles With histogram_quantile (2 labs): You can read cumulative histogram buckets and use histogram quantile to estimate a percentile without pretending that an average describes the tail.
  4. Grafana: Connecting The Data Source (2 labs): You can connect Grafana to Prometheus, run a query in Explore, and distinguish a data source problem from a query problem.
  5. Dashboards That Answer Questions (2 labs): You can design a focused dashboard around a user facing question, choose panels that support it, and remove decoration that cannot guide a decision.

OpenTelemetry And Distributed Tracing

  1. OpenTelemetry: The Vendor Neutral Path (1 lab): You can explain why OpenTelemetry separates instrumentation from storage vendors and identify the common language shared by traces, metrics, and logs.
  2. Traces, Spans And Context Propagation (1 lab): You can read a trace tree, distinguish a parent span from a child span, and explain how trace context crosses a service boundary.
  3. The OpenTelemetry Collector (1 lab): You can read a collector pipeline, explain receivers processors and exporters, and describe why batching and memory limits belong at the boundary.
  4. Instrumenting A Service With Client Libraries (1 lab): You can identify the instrumentation points in a real service and explain what a client library should record automatically versus what your code must name explicitly.
  5. Bringing The Three Signals Together (2 labs): You can move from a metric symptom to a trace and then to a structured log by following shared service and request context.

Alerting And Reliability

  1. Alerting On Symptoms Versus Causes (1 lab): You can choose alerts that represent customer impact and use causes as investigation evidence.
  2. Alerting Rules And The For Duration (2 labs): You can write a Prometheus alert condition with a for duration and explain why persistence filters noise.
  3. Service Level Indicators (3 labs): You can define an availability SLI and a latency SLI as good events over eligible events, write each as a Prometheus ratio, and choose where to measure it.
  4. Service Level Objectives (1 lab): You can state an SLO as an indicator, a target and a window, convert a target into allowed failure time, choose between rolling and calendar windows, and pick a target from evidence rather than ambition.
  5. Error Budgets And Burn Rate Alerts (3 labs): You can calculate an error budget from an SLO, express the current error rate as a burn rate, and write multiwindow, multi burn rate alerts that page for fast and slow budget loss.

Incident Response And Postmortems

  1. On Call And Incident Command (2 labs): You can establish an incident commander, separate coordination from investigation, and keep a reliable timeline during an outage.
  2. The Blameless Postmortem (2 labs): You can write a blameless postmortem that explains contributing conditions, impact, detection, response, and concrete follow up work.
  3. A Worked Example Postmortem (2 labs): You can use a worked outage to connect symptoms, evidence, response decisions, and follow up actions in a complete postmortem.

Certification courses