rate limiting
Capping how many requests a client may make in a period of time, and rejecting or delaying the rest.
In more detail
A policy has a scope, such as a user, key, address or resource, and a limit per time window or a sustained rate with room for short bursts. Servers use it to protect shared capacity from overload and abuse. HTTP APIs reply 429 Too Many Requests, often with a Retry-After header, and clients should back off before trying again.
Examples
A simple sliding-window limiter keeps a deque of recent request times, drops those older than the window from the left, and refuses a request when the deque is full.X-RateLimit-Limit: 2HTTP/1.1 429 Too Many RequestsRetry-After: 1Related terms
- sliding window — The most recent k items of a sequence or stream, which moves forward one item at a time as new data arrives.
- deque — A double-ended queue: a sequence that adds and removes items at either end in O(1) time; Python provides it as collections.deque.
- middleware — Code that sits between an incoming request and the handler that answers it, doing a shared job such as logging, authentication or rate limiting.
- denial of service — An attack that makes a service unavailable to its real users by overloading it or making it crash.
- 429 Too Many Requests — The HTTP status meaning the caller has sent too many requests in a given time and should slow down.
- Retry-After — The response header that tells a client how long to wait, in seconds or as a date, before trying again.
- RateLimit header fields — Response headers that tell a client its rate-limit budget: the limit, how much remains and when it resets.
- fixed window — A rate-limiting method that counts requests in fixed blocks of time, such as two per second, resetting the count when each block ends.
- quota — A total allowance of use over a long period, such as a number of requests per day or month.
- token bucket — A rate limiting algorithm in which requests spend tokens from a bucket that refills at a steady rate, allowing short bursts up to the bucket size.
- rate limit partition — A separate rate limit counter kept per key, such as per user or per client address, so one caller's usage does not use up another's.
- sliding window counter — A rate-limiting method that estimates the requests in the last full window by weighting the previous window’s count by how much of it still overlaps.
- middleware.ts — A single file at the project root that runs code before matching requests are routed, to redirect, rewrite, set headers or check sessions.
- higher-order function — A function that takes a function as an argument or returns one, such as map, filter or a wrapper that adds retries.
- brute force attack — Trying many passwords against one account until one works.
- backoff — Waiting longer and longer between successive retries, often doubling the delay each time, so a struggling service has room to recover.
- unrestricted resource consumption — OWASP's name for APIs that let callers spend unlimited processor time, memory, connections, rows or money.
- throttling — A service refusing or delaying requests because a rate or concurrency limit has been reached.
- admission control — Deciding at the entrance whether a request may start, based on how much work the system can still take.
Taught in
- What An AI Engineer Does — AI Engineering: LLM Apps, RAG & Agents
- Streaming Responses — AI Engineering: LLM Apps, RAG & Agents
- When The Call Fails — AI Engineering: LLM Apps, RAG & Agents
- Excessive Agency — AI Engineering: LLM Apps, RAG & Agents
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Choosing The Right Sequence — Algorithms & Data Structures for Working Engineers
- Rate Limiting Strategies — Modern API Architecture: REST, GraphQL & gRPC
- API Gateways And Proxies — Modern API Architecture: REST, GraphQL & gRPC
- The ASP.NET Core Middleware Pipeline — Modern .NET Core, C# & Enterprise Microservices
- Building Routes with Minimal APIs — Modern .NET Core, C# & Enterprise Microservices
- Built-in Rate Limiting Middleware — Modern .NET Core, C# & Enterprise Microservices
- Type-Safe RPC Architectures: tRPC & Procedures — Production Full-Stack Engineering
- API Gateway & BFF Patterns: Backend-for-Frontend — Production Full-Stack Engineering
- Traffic Management: Sliding Window Rate Limiting — Production Full-Stack Engineering
- Server Action Security: Zod Validation & Auth Gates — Next.js 15 & Modern App Router Architecture
- Authentication Versus Authorisation — Application Security & Threat Modeling for Engineers
- Rate Limiting And Resource Consumption — Application Security & Threat Modeling for Engineers
- The Storage Decision: Key Value, Search And Blob — Scalable System Design & Distributed Architecture
- Worked Design Three: A Metrics Ingestion Pipeline — Scalable System Design & Distributed Architecture
References
- RFC 6585: 429 Too Many Requests
- IETF draft: RateLimit header fields for HTTP
- OWASP API Security Top 10 (2023)
- Rate limiting middleware in ASP.NET Core
- Azure Architecture Center: Rate Limiting pattern
- MDN: 429 Too Many Requests
- OWASP Denial of Service Cheat Sheet
- OWASP API4:2023 Unrestricted Resource Consumption
- Envoy documentation: Global rate limiting