tail latency
The latency of the slowest requests, measured by high percentiles such as p99.
In more detail
Users often feel the tail more than the average, and one slow dependency in a chain of calls pushes more requests into it. Queueing, garbage collection pauses and lock contention are common causes.
Examples
A page that makes 20 requests meets the slowest 1 per cent about 18 per cent of the time.If one call in a hundred is slow, a request that fans out to a hundred calls is slow about 63 percent of the time.In this course: A fast average with a growing tail is a warning sign. The tail is what real users feel on a bad request, so it is usually what a service objective should promise.Related terms
- percentile — The value below which a given percentage of measurements fall; p99 latency is the time 99 in 100 requests beat.
- latency — The time between asking for something and getting the answer back.
- mean — The sum of all values divided by how many there are; what people usually call the average.
- service level indicator — A measurement of how well a service is doing for users, usually good events divided by eligible events.
- fan-out — Sending one request or message on to many recipients or services at once.
- hedged request — Sending a second copy of a slow request to another server and using whichever answer arrives first.
- service level objective — A target for a service level indicator over a time window, such as 99.9 per cent of checkouts succeeding over 30 days.
Taught in
- Load Testing — Modern API Architecture: REST, GraphQL & gRPC
- Performance Optimization — Modern API Architecture: REST, GraphQL & gRPC
- Throughput: Change Lead Time And Deployment Frequency — DevOps & Site Reliability Engineering
- Percentiles Rather Than Averages — DevOps & Site Reliability Engineering
- Continuous Delivery: Blue-Green & Canary Pipelines — Production Full-Stack Engineering
- Calculating Percentiles With histogram_quantile — Full-Stack Observability: Metrics, Tracing & Logging
- Requirements: The Constraints That Shape The System — Scalable System Design & Distributed Architecture
- Latency Numbers Every Engineer Should Know — Scalable System Design & Distributed Architecture
- Load Testing: Percentiles Over Means — Automated Testing, TDD & Quality Engineering