fencing
Cutting a failed or doubtful member of a replicated system off from its data so it cannot keep writing while you recover.
In more detail
If two copies both think they are primary, both can write and corrupt the data. Stopping writes or fencing the failed member is therefore the first step of recovery, before restoring or reopening traffic.
Related terms
- disaster recovery — The planned, tested procedure for getting a system and its data back after a serious failure, such as lost storage or a deleted cluster.
- StatefulSet — The Kubernetes object for replicated applications where each pod needs a stable name and its own storage, such as databases.
- leader epoch — A counter per partition that goes up by one every time a new leader is elected.
- distributed-lease — A lock that expires automatically after a set time unless the holder renews it.
- split brain — A failure in which two nodes both believe they are the leader and both accept writes.
- failover — Switching work from a failed or unavailable system to a standby copy so the service keeps running.
Taught in
- StatefulSets, Stable Identity And Data Recovery — Kubernetes: Production-Grade Container Orchestration
- Replication, Failover And Leader Election — Scalable System Design & Distributed Architecture
- Partitioning, Sharding And Hot Keys — Scalable System Design & Distributed Architecture