bounded concurrency
Capping how many requests a caller will have in progress to a dependency at once.
In more detail
It stops a slow dependency from consuming every worker and turns overload into fast, explicit refusals. Requests beyond the limit are queued briefly or refused quickly.
Related terms
- cascading failure — A failure in one part of a system that overloads or breaks the parts around it, spreading until much more is down.
- concurrency — Having several pieces of work in progress during overlapping periods of time, such as requests being handled at once.
- backpressure — A slow consumer signalling a fast producer to pause, so data does not pile up in memory faster than it can be handled.
Taught in
- Timeouts, And Avoiding Cascaded Failure — Scalable System Design & Distributed Architecture