trace
A record of one request’s whole journey through a system, made of nested spans that show where the time went.
In more detail
A gateway span can contain a service span, which contains a database or queue span. Trace context must be propagated across service and queue boundaries for the pieces to join up.
Example
GET /checkout (root) with children SELECT basket and POST payments, all sharing trace ID 216363698b529b4a97b750923ceb3ffd.Related terms
- span — One timed operation within a trace, such as handling a request or running a query, with its start, duration and attributes.
- Trace Context — The standard way of passing a trace identifier and parent span identifier from one service to the next, usually in the traceparent header.
- sampling — Recording only some traces to save cost, chosen by a rule such as a percentage or by keeping every error and slow request.
- observability — How well you can understand what a system is doing, and why, from the data it emits: mainly logs, metrics and traces.
- correlation ID — A unique value attached to one request and passed along everywhere it goes, so its logs and traces can be found together.
- metric — A number measured over time, such as requests per second or error count, that shows how a system behaves in aggregate.
- log — A time-stamped record of an individual event in a system, such as one request being handled; structured logs record it as named fields rather than free text.
- OpenTelemetry — A vendor-neutral, open-source standard and set of tools for producing traces, metrics and logs and sending them to any backend.
- trace ID — An identifier shared by every span and log line belonging to one request, so they can be found together.
- distributed tracing — Following one request across every service, queue and database it touches by recording linked spans that share a trace identifier.
Taught in
- What An AI Engineer Does — AI Engineering: LLM Apps, RAG & Agents
- Memory And Compaction — AI Engineering: LLM Apps, RAG & Agents
- Designing Tools A Model Can Use — AI Engineering: LLM Apps, RAG & Agents
- Planning, Memory And Many Agents — AI Engineering: LLM Apps, RAG & Agents
- Agent SDKs And Human Approval — AI Engineering: LLM Apps, RAG & Agents
- Threat-Model An LLM App — AI Engineering: LLM Apps, RAG & Agents
- Excessive Agency — AI Engineering: LLM Apps, RAG & Agents
- Regression Tests In CI — AI Engineering: LLM Apps, RAG & Agents
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Capstone: The Tasks Assistant — AI Engineering: LLM Apps, RAG & Agents
- Problem Details: RFC 9457 — Modern API Architecture: REST, GraphQL & gRPC
- Observability: Logs, Metrics, Traces — Modern API Architecture: REST, GraphQL & gRPC
- Ten Findings, Six Pillars — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Four Hundred Missing Labels — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Two Hundred Accounts And One Bad Deploy — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Collecting Logs And Metrics: The Disk Nobody Measured — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Finding The Problem: The Composite Alarm That Stayed Quiet — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Incident Tooling: Eighteen Minutes To Find A Human — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Virtual Private Clouds — Cloud Infrastructure & Hyperscaler Architecture
- Route Tables and Internet Gateways — Cloud Infrastructure & Hyperscaler Architecture
- Security Groups and Firewalls — Cloud Infrastructure & Hyperscaler Architecture
- Load Balancing and DNS — Cloud Infrastructure & Hyperscaler Architecture
- Serverless Compute (Lambda) — Cloud Infrastructure & Hyperscaler Architecture
- Observability with CloudWatch — Cloud Infrastructure & Hyperscaler Architecture
- Metrics, Logs, and Alarms — Cloud Infrastructure & Hyperscaler Architecture
- How Bills Actually Get Large — Cloud Infrastructure & Hyperscaler Architecture
- Observability Context — DevOps & Site Reliability Engineering
- Observability Versus Monitoring — Full-Stack Observability: Metrics, Tracing & Logging
- The Three Signals: Logs, Metrics And Traces — Full-Stack Observability: Metrics, Tracing & Logging
- Structured Logging And Why Text Fails — Full-Stack Observability: Metrics, Tracing & Logging
- Cardinality And How It Bankrupts You — Full-Stack Observability: Metrics, Tracing & Logging
- The Prometheus Exposition Format — Full-Stack Observability: Metrics, Tracing & Logging
- Dashboards That Answer Questions — Full-Stack Observability: Metrics, Tracing & Logging
- OpenTelemetry: The Vendor Neutral Path — Full-Stack Observability: Metrics, Tracing & Logging
- Traces, Spans And Context Propagation — Full-Stack Observability: Metrics, Tracing & Logging
- The OpenTelemetry Collector — Full-Stack Observability: Metrics, Tracing & Logging
- Instrumenting A Service With Client Libraries — Full-Stack Observability: Metrics, Tracing & Logging
- Bringing The Three Signals Together — Full-Stack Observability: Metrics, Tracing & Logging
- A Worked Example Postmortem — Full-Stack Observability: Metrics, Tracing & Logging
- Fifteen Traces For Six Bookings: Where Context Was Lost — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Ninety Minutes, No Question Count: Exam Strategy — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Traces From A Script Nobody Touched: Instrumentation Approaches — OpenTelemetry Certified Associate (OTCA) Exam Prep
- A Warning With An Address: Correlating Signals — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Fundamentals Sprint: One Declined Card — OpenTelemetry Certified Associate (OTCA) Exam Prep
- A Library That Traces Nothing, Safely: API, SDK And Extension — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Fifty Two Spans, Four Requests: The Trace Data Model — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Three Hundred Milliseconds In One Place: Tracing In Code — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Keep Your Logger: The Logs Bridge And Log Records — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Fifty Five Characters: Trace Context And Baggage — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Six Out Of Twelve At One In Four: Samplers — OpenTelemetry Certified Associate (OTCA) Exam Prep
- The Spans Of A Job That Ended Too Soon: SDK Pipelines — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Nine Lines, Five Bugs: Configuring The SDK — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Six Launches, No Code Changes: Language Agents — OpenTelemetry Certified Associate (OTCA) Exam Prep
- API And SDK Sprint: Fourteen Predictions — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Sixty Two New Names: Collector Components And Pipelines — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Validated And Deaf: Checking A Collector Config — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Fifty Five In, Fifty Two Out: Transforming Data — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Which Node Was It? Collector Deployment Patterns — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Two Replicas, Half A Trace: Scaling The Collector — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Collector Sprint: Sixteen Predictions — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Maintenance Sprint: Symptoms First — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Forty Four Of Sixty: The Mock And What It Tells You — OpenTelemetry Certified Associate (OTCA) Exam Prep
- The Evening Before: Final Review — OpenTelemetry Certified Associate (OTCA) Exam Prep
- Six Minutes Nobody Saw: Prometheus In One Outage — Prometheus Certified Associate (PCA) Exam Prep
- Ninety Minutes, No Question Count: Exam Strategy — Prometheus Certified Associate (PCA) Exam Prep
- Observability Sprint: One Failed Checkout — Prometheus Certified Associate (PCA) Exam Prep
- Off By Five: What Prometheus Is Not For — Prometheus Certified Associate (PCA) Exam Prep
- Fundamentals Sprint: Nine Calls On The Server — Prometheus Certified Associate (PCA) Exam Prep
- Milliseconds Versus Seconds: Naming Metrics And Labels — Prometheus Certified Associate (PCA) Exam Prep
- One Week, Three Objectives: Final Review — Prometheus Certified Associate (PCA) Exam Prep
- Timeouts, And Avoiding Cascaded Failure — Scalable System Design & Distributed Architecture
- Monolith Versus Services: The Honest Trade Offs — Scalable System Design & Distributed Architecture
- Observability As A Design Input — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture
References
Certification courses
- AWS AIP-C01 course and practice exam: AI engineering with LLM apps, RAG and agents
- AWS Cloud Practitioner course and practice exam: CLF-C02
- AWS DevOps Engineer Professional course and practice exam: DOP-C02
- OTCA course and practice exam: OpenTelemetry Certified Associate
- PCA course and practice exam: Prometheus Certified Associate