latency budget
A latency target divided among the hops a request makes, so each part knows how much time it may spend.
In more detail
Build it by subtraction from the user promise: list each hop, add up the costs, and see which hop to attack. Budget at the tail, not at typical numbers, and leave a margin for variance.
Example
200 ms budget: load balancer 1, app work 20, cache read 1, database read 8, cross region call 80.Related terms
- deadline — The point in time by which a request must finish, after which its answer is no longer useful.
- hop — One step a request takes between two machines or programs on its way to the final server.
- tail latency — The latency of the slowest requests, measured by high percentiles such as p99.
- deadline propagation — Passing a request's absolute deadline to every service it calls, so each knows how much time is left.
Taught in
- Latency Numbers Every Engineer Should Know — Scalable System Design & Distributed Architecture