load balancing
Spreading requests or connections across several backend servers so no single one is overloaded and failed ones can be avoided.
In more detail
A load balancer picks a backend for each request using an algorithm such as round robin or least connections, and uses health checks to decide which backends are eligible. It can work at the transport layer, seeing only connections, or at the application layer, seeing HTTP requests. Uneven traffic often comes from affinity or long-lived connections rather than a broken algorithm.
Example
In Kubernetes only ready pods receive traffic.Related terms
- stateless process — A process that keeps nothing it needs between requests in its own memory or disk, so it can be restarted or replaced at any moment.
- blue-green deployment — Running two complete environments, blue for the current release and green for the candidate, and switching traffic between them in one step.
- horizontal scaling — Handling more load by running more copies of a process, often on more machines, rather than making one machine bigger.
- router — A device or service that decides where network traffic is sent.
- readiness probe — A check that decides whether a pod should receive traffic right now; failing it removes the pod from its Service without restarting it.
- forwarded headers — HTTP headers such as X-Forwarded-For that a proxy adds to tell the app the original client address and scheme.
- reverse proxy — A server that receives requests on behalf of other servers and forwards them, returning their responses as if they were its own.
- backend pool — The set of servers a load balancer can send traffic to, each marked eligible or not by health checks.
- round robin — A way of spreading work by taking turns in a fixed order, giving the next item to the next recipient in the list.
- least connections — A load-balancing algorithm that sends each new request to the backend currently handling the fewest active connections.
- session affinity — Sending all requests from one client to the same backend, often by hashing the client address or using a cookie.
- load balancer health check — A test a load balancer runs against each backend, such as opening a port or requesting a URL, to decide whether it should receive traffic.
- health check — A regular automated test of whether a program or service is working, such as a request to its health endpoint or a command run inside its container.
Taught in
- Build, Release, Run And Processes — DevOps & Site Reliability Engineering
- Deployment Strategies: Blue Green — DevOps & Site Reliability Engineering
- Health Checks — Modern .NET Core, C# & Enterprise Microservices
- Server-Sent Events: Streaming Updates & Recovery — Production Full-Stack Engineering
- Health Probes & Graceful Shutdown: Kubernetes Lifecycles — Production Full-Stack Engineering
- What HTTPS Adds, And What It Does Not — Linux Fundamentals & Systems Administration
- Load Balancing: Algorithms, Health Checks And Layers — Linux Fundamentals & Systems Administration
- Cluster Architecture: node:cluster & Zero-Downtime — Node.js Internals & Backend Services