load balancer
A server that spreads incoming requests across several copies of a service so no single copy is overloaded.
In more detail
Stateless requests let it send each request to any healthy copy without sticky sessions. It checks the health of each copy and stops sending traffic to ones that fail.
Examples
In AWS: A service that receives connections at one stable address and spreads them across several healthy servers behind it.It keeps a pool of backends, picks one for each request or connection with an algorithm such as round robin or least connections, and runs health checks to skip backends that fail. A transport-layer balancer sees only connections; an application-layer balancer reads HTTP requests and can terminate TLS. It is itself an intermediary, so trace the client-to-balancer and balancer-to-backend legs separately.algo=rr req=42 be=app-a ok=y shows a balancer choosing a healthy backend by round robin.Related terms
- stateless — Describes a component or service that keeps no data from one request to the next, so each request carries everything needed to handle it.
- sticky session — A load balancer setting that sends every request from one client to the same server copy, usually because that copy holds its state in memory.
- reverse proxy — A server that receives requests on behalf of other servers and forwards them, returning their responses as if they were its own.
- load balancer listener — The part of a load balancer that accepts connections on one protocol and port and decides which target group gets them.
- target group — The set of servers, containers or functions a load balancer forwards requests to, with the health check that judges them.
- service health check — A regular automatic test, run by a load balancer, Auto Scaling group or container service, of whether a copy of a service can really handle requests.
- horizontal scaling — Handling more load by running more copies of a process, often on more machines, rather than making one machine bigger.
- blue-green deployment — Running two complete environments, blue for the current release and green for the candidate, and switching traffic between them in one step.
- traffic routing — Deciding which version of a service receives which share of requests.
- load balancing — Spreading requests or connections across several backend servers so no single one is overloaded and failed ones can be avoided.
- backend pool — The set of servers a load balancer can send traffic to, each marked eligible or not by health checks.
- load balancer health check — A test a load balancer runs against each backend, such as opening a port or requesting a URL, to decide whether it should receive traffic.
- TLS termination — Ending an encrypted TLS connection at a proxy or gateway, which decrypts the traffic and forwards it on to the service.
- health check — A regular automated test of whether a program or service is working, such as a request to its health endpoint or a command run inside its container.
- round robin — A way of spreading work by taking turns in a fixed order, giving the next item to the next recipient in the list.
- least connections — A load-balancing algorithm that sends each new request to the backend currently handling the fewest active connections.
- transport layer — The layer that carries data between programs on two machines, identified by port numbers, using TCP or UDP.
- application layer — The top layer, where protocols such as HTTP, DNS and SSH give the bytes meaning as requests, answers and commands.
Taught in
- What An API Is, And The Alternatives — Modern API Architecture: REST, GraphQL & gRPC
- The REST Constraints — Modern API Architecture: REST, GraphQL & gRPC
- Twelve Alerts, Ten Services — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Which Zone, Which Journey? — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Nine Jobs, Not Nine Servers — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Billing Sprint: Worth An Evening? — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Confusable Services Sprint: One Clue Each — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Two Hundred Accounts And One Bad Deploy — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- CodeDeploy: Hooks That Never Ran — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Strategies Across Compute: The Canary That Saw Nothing — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Systems Manager At Fleet Scale: Compliant And Unpatched — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- High Availability Across Tiers: What Fails Over By Itself? — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Monitoring As Code: The Alarm That Watched Nothing — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Incident Tooling: Eighteen Minutes To Find A Human — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Security Hub, GuardDuty And Inspector: Signal, Noise And Response — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Long-Stem Clinic: When Two Answers Both Work — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The Morning An Availability Zone Went Dark — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Network As A Control: CIDR, Routes, ACLs And Security Groups — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Private Paths: VPC Endpoints And PrivateLink — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Edge: WAF, Shield, CloudFront And GuardDuty — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- KMS And Envelope Encryption: Who Can Still Decrypt? — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Serverless And Container Patterns: Where The Limits Bite — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Load Balancers And Auto Scaling: Capacity That Arrives On Time — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Route 53 Routing: Where Does A Fan In Lagos Land? — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Performance Clinic: Five Slow Systems, Five Fixes — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Cost Governance: Tags, Budgets And Reports That Speak Up — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Long-Stem Clinic: The Phrase That Decides — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Monday, Five Tickets — Microsoft Azure Administrator (AZ-104) Exam Prep
- Ninety-Nine Percent Full: Disks, Sizes And Availability — Microsoft Azure Administrator (AZ-104) Exam Prep
- Monday At Eight: Scale Sets And Autoscale — Microsoft Azure Administrator (AZ-104) Exam Prep
- Port 22 Answers From The Internet: NSGs And Application Security Groups — Microsoft Azure Administrator (AZ-104) Exam Prep
- No Public Doors: Bastion, Service Endpoints And Private Endpoints — Microsoft Azure Administrator (AZ-104) Exam Prep
- Healthy, Said The Load Balancer: Azure DNS And Load Balancing — Microsoft Azure Administrator (AZ-104) Exam Prep
- Blue-Green Deployments — Continuous Integration & Continuous Delivery Pipelines
- Rolling Back Blue-Green — Continuous Integration & Continuous Delivery Pipelines
- Regions, Availability Zones and Reliability — Cloud Infrastructure & Hyperscaler Architecture
- Subnets: Public and Private — Cloud Infrastructure & Hyperscaler Architecture
- Security Groups and Firewalls — Cloud Infrastructure & Hyperscaler Architecture
- Load Balancing and DNS — Cloud Infrastructure & Hyperscaler Architecture
- Containers on AWS (ECS and EKS) — Cloud Infrastructure & Hyperscaler Architecture
- Auto Scaling Groups — Cloud Infrastructure & Hyperscaler Architecture
- The Two A.M. Page — CompTIA Cloud+ (CV0-004) Exam Prep
- The Range That Already Belonged To Someone — CompTIA Cloud+ (CV0-004) Exam Prep
- Peered To Everything, Reaching Nothing — CompTIA Cloud+ (CV0-004) Exam Prep
- Seven Wishes: AI Services And The Internet Of Things — CompTIA Cloud+ (CV0-004) Exam Prep
- A Worksheet With Eight Boxes: Provisioning To Requirements — CompTIA Cloud+ (CV0-004) Exam Prep
- Nine In The Morning, Every Monday: Scaling — CompTIA Cloud+ (CV0-004) Exam Prep
- An Image That Runs As Root: Security Best Practices — CompTIA Cloud+ (CV0-004) Exam Prep
- The Bill That Jumped Overnight: Spotting Attacks — CompTIA Cloud+ (CV0-004) Exam Prep
- Five Deploys, Five Failures: Troubleshooting Deployments — CompTIA Cloud+ (CV0-004) Exam Prep
- Fine From Home, Broken At The Office: Network Troubleshooting — CompTIA Cloud+ (CV0-004) Exam Prep
- Troubleshooting Sprint: Three Tickets, One Morning — CompTIA Cloud+ (CV0-004) Exam Prep
- Exam On Thursday: The PBQ Clinic And Final Review — CompTIA Cloud+ (CV0-004) Exam Prep
- What HTTPS Adds, And What It Does Not — Linux Fundamentals & Systems Administration
- Forward And Reverse Proxies — Linux Fundamentals & Systems Administration
- Load Balancing: Algorithms, Health Checks And Layers — Linux Fundamentals & Systems Administration
- TLS And The Handshake — Application Security & Threat Modeling for Engineers
- Latency Numbers Every Engineer Should Know — Scalable System Design & Distributed Architecture
- Load Balancing And The Failure Modes It Introduces — Scalable System Design & Distributed Architecture
- The Plan That Wanted To Destroy Production — HashiCorp Terraform Associate (004) Exam Prep
References
- NGINX docs: HTTP load balancing
- AWS: What is Elastic Load Balancing?
- What is an Application Load Balancer?
- Kubernetes: Service
- NGINX: using nginx as HTTP load balancer
- HAProxy documentation
- Google SRE book: Load Balancing in the Datacenter
Certification courses
- AWS Cloud Practitioner course and practice exam: CLF-C02
- AWS DevOps Engineer Professional course and practice exam: DOP-C02
- AWS Solutions Architect Associate course and practice exam: SAA-C03
- AZ-104 course and practice exam: Microsoft Azure Administrator
- CompTIA Cloud+ course and practice exam: CV0-004
- Terraform Associate course and practice exam: HashiCorp Terraform Associate (004)