retry budget
A cap on retries measured as a fraction of normal traffic, so retries cannot overwhelm a failing dependency.
In more detail
An unhealthy dependency cannot be healed by unlimited demand. One budget, owned by the outermost layer, is easier to measure and turn down during an incident.
Related terms
- retry — Sending a request again after it failed or timed out, in the hope that it succeeds the second time.
- retry storm — A flood of retries from many clients at once that hits an already struggling service harder than the original traffic.
- exponential backoff — Waiting progressively longer between retries, for example 1, 2, then 4 seconds, so a struggling service gets room to recover.
Taught in
- Retries, Backoff And Jitter — Scalable System Design & Distributed Architecture