single flight
Letting only one request fetch a missing value while other requests for the same key wait for its result.
In more detail
It turns a herd of misses on one hot key into a single origin call. It is usually combined with serving the stale value while the one worker refreshes.
Example
In the simulation, 30 origin calls without single flight become 4 with it.Related terms
- thundering herd — A flood of requests that arrives at once when many clients react to the same event, such as all retrying after an outage.
- stale-while-revalidate — A caching strategy that answers immediately from the cache, even if out of date, while fetching a fresh copy in the background for next time.
- lock — A marker that gives one worker exclusive access to a resource until it releases it.
Taught in
- Cache Invalidation And The Three Hard Cases — Scalable System Design & Distributed Architecture