queueing
Waiting in line for a busy resource, such as a thread, connection or disk.
In more detail
Queueing delay is not linear: as utilisation approaches one hundred percent the wait grows without limit. That is why headroom is necessary and why the tail of a latency distribution is where queueing lives.
Related terms
- queueing theory — The mathematics of waiting lines, describing how arrival rates and service times produce delay.
- knee of the curve — The point where a small rise in load starts causing a large rise in waiting time.
- utilization — The fraction of time a resource is busy.
- Little's law — A rule from queueing theory: average lead time equals average work in progress divided by average throughput.
Taught in
- Latency Numbers Every Engineer Should Know — Scalable System Design & Distributed Architecture
- Partitioning, Sharding And Hot Keys — Scalable System Design & Distributed Architecture
- Timeouts, And Avoiding Cascaded Failure — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture