leader election
The process by which the members of a distributed system choose one of themselves to act as leader.
In more detail
Elections run automatically when the current leader fails, so the system can keep accepting work. Protocols such as Raft make sure only one leader is recognised at a time.
Examples
Kafka elects a new partition leader from the in-sync replicas, and each election raises the leader epoch.The Kubernetes scheduler and controller manager elect a leader by renewing a Lease object.Related terms
- Lease — A small Kubernetes object that a component keeps renewing to show it is alive or that it holds a leadership role.
- high availability — Designing a system so it keeps working when one part fails, usually by running redundant copies with no single point of failure.
- kube-controller-manager — The control plane component that runs the loops keeping the cluster's actual state matching the desired state.
- Raft — An agreement method that lets several copies of a database accept writes in the same order and keep working when a member fails.
- partition leader — The replica of a partition that handles all reads and writes for it; the other replicas copy from it.
- leader epoch — A counter per partition that goes up by one every time a new leader is elected.
- in-sync replica set — The replicas of a partition that are currently caught up with the leader, abbreviated ISR.
- Kafka controller — The broker currently responsible for cluster-wide decisions such as tracking brokers and choosing partition leaders.
- leader — The one replica that accepts writes and decides their order, which followers then copy.
- distributed-lease — A lock that expires automatically after a set time unless the holder renews it.
- fencing — Cutting a failed or doubtful member of a replicated system off from its data so it cannot keep writing while you recover.
- coordination service — A small, highly reliable service, such as etcd or ZooKeeper, that other systems use for locks, leases and leader election.
Taught in
- HA Control Planes: Quorum, Stacked Or External — Certified Kubernetes Administrator (CKA) Exam Prep
- Acknowledged Is Not Always Safe — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The New Broker That Did Nothing — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty One Under-Replicated Partitions — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Upgrade That Could Not Start — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Controllers At Once — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Three Brokers: Replication And Leaders — Apache Kafka & Event Streaming
- Losing A Broker — Apache Kafka & Event Streaming
- kubeadm, High Availability And Cluster Lifecycle — Kubernetes: Production-Grade Container Orchestration
- Replication, Failover And Leader Election — Scalable System Design & Distributed Architecture
References
- Apache Kafka design: replication
- Amazon Builders' Library: Leader election in distributed systems
- Azure Architecture Center: Leader Election pattern
- The Raft Consensus Algorithm