load shedding
Deliberately rejecting some requests when overloaded so the requests that are accepted can still be served well.
In more detail
Serving some users well is better than serving everyone badly. Shed requests get a fast error, often 429 or 503, instead of a slow timeout.
Example
An open circuit breaker sheds load from a failing dependency.Related terms
- circuit breaker — A wrapper around calls to another service that stops making calls for a while after repeated failures, so the failing service can recover.
- rate limiting — Capping how many requests a client may make in a period of time, and rejecting or delaying the rest.
- admission control — Deciding at the entrance whether a request may start, based on how much work the system can still take.
- backpressure — A slow consumer signalling a fast producer to pause, so data does not pile up in memory faster than it can be handled.
- graceful degradation — Continuing to work with reduced features when part of the system fails, instead of failing completely.
Taught in
- Resilience Engineering: Circuit Breakers & Retries — Production Full-Stack Engineering
- Queues, Events And Asynchronous Work — Scalable System Design & Distributed Architecture
- Timeouts, And Avoiding Cascaded Failure — Scalable System Design & Distributed Architecture
- Backpressure And Load Shedding — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture
- Worked Design One: Scaling The Ticket Booking System — Scalable System Design & Distributed Architecture
- Worked Design Three: A Metrics Ingestion Pipeline — Scalable System Design & Distributed Architecture
- Worked Design Three: Scaling The Metrics Pipeline — Scalable System Design & Distributed Architecture