replication
Keeping copies of data in more than one place, updated automatically, so one failure does not lose or block it.
In more detail
Replication improves durability, availability and sometimes read speed. It is not a backup: deletions and corruption are copied to every replica too.
Examples
Kafka replicates each partition to several brokers; the replication factor sets how many copies exist.ship(primary, near) # copy rows the replica has not seen yet (log shipping in a few lines)Related terms
- durability — The likelihood that stored data survives hardware failures without being lost, usually stated as a percentage per year.
- read replica — A copy of a database that follows the primary’s changes and can serve read traffic or be promoted to take over.
- backup and restore — Keeping recoverable copies of data from past points in time, and being able to bring them back.
- partition replica — One copy of a partition stored on one broker.
- replication factor — The number of copies Kafka keeps of each partition, including the leader.
- partition leader — The replica of a partition that handles all reads and writes for it; the other replicas copy from it.
- follower — A replica that copies a partition's records from its leader and can take over if the leader fails.
- synchronous replication — Replication in which a write is only acknowledged after another copy has durably stored it.
- asynchronous replication — Replication in which a write is acknowledged first and copied to other machines afterwards.
- hot site — A fully equipped recovery site running the same systems as the main site with continuously replicated data, ready to take over within minutes.
- backup — An extra copy of data kept separately so it can be restored if the original is lost, damaged, deleted or encrypted by an attacker.
- replica — A copy of a dataset kept on another machine and updated from the original.
- replication factor — The number of copies of each piece of data a system keeps, commonly three.
- replication lag — How far behind the leader a replica is, measured in time or in writes.
- leader — The one replica that accepts writes and decides their order, which followers then copy.
Taught in
- Seven Ways To Leave A Rack — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Cloud Concepts Sprint: Twelve Cards — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- One Database Doing Five Jobs — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Ten Terabytes Nobody Reads — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Protecting Data: Versions, Object Lock, Backups And Secrets — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Databases That Survive: Standbys, Replicas And Global Tables — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Disaster Recovery By The Numbers — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Storage Performance: IOPS, Throughput And The Right Volume — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Database Performance: Caches, Proxies And Capacity Units — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Database Cost: Serverless, Capacity Modes And Reservations — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Network Cost: NAT Gateways, Zones And Data Out — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Long-Stem Clinic: The Phrase That Decides — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Shared Responsibility Model — Cloud Infrastructure & Hyperscaler Architecture
- Object, Block, and File Storage — Cloud Infrastructure & Hyperscaler Architecture
- When to Pick Which Database — Cloud Infrastructure & Hyperscaler Architecture
- How Bills Actually Get Large — Cloud Infrastructure & Hyperscaler Architecture
- Who Patches What: Service Models And Availability — CompTIA Cloud+ (CV0-004) Exam Prep
- Peered To Everything, Reaching Nothing — CompTIA Cloud+ (CV0-004) Exam Prep
- Forty Terabytes Nobody Opens: Storage And Databases — CompTIA Cloud+ (CV0-004) Exam Prep
- The Container That Did Not Come Back — CompTIA Cloud+ (CV0-004) Exam Prep
- Two Domain Controllers, One Host: Virtualisation — CompTIA Cloud+ (CV0-004) Exam Prep
- Architecture Sprint: Ten Faults Before Friday — CompTIA Cloud+ (CV0-004) Exam Prep
- A Release That Cannot Take Checkout Down — CompTIA Cloud+ (CV0-004) Exam Prep
- Slow At Lunchtime: Logs, Traces And Alerts — CompTIA Cloud+ (CV0-004) Exam Prep
- Thursday, Twenty Past Two: Backup And Recovery — CompTIA Cloud+ (CV0-004) Exam Prep
- Sixty Six Of Eighty Five: The Mock And Its Report — CompTIA Cloud+ (CV0-004) Exam Prep
- Exam On Thursday: The PBQ Clinic And Final Review — CompTIA Cloud+ (CV0-004) Exam Prep
- The Payment That Was Charged Twice — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Ninety Minutes, No Question Count — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Where A Record Lands — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Acknowledged Is Not Always Safe — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Blank Names After Every Deploy — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Setting That Came Back — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The New Broker That Did Nothing — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty Minutes To Start — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Data Centre That Went Dark — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Full Disks On Black Friday — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Controllers At Once — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Disaster Recovery Sites: Hot, Warm & Cold — Cybersecurity Fundamentals, GRC & Cryptography
- Backup Architectures: 3-2-1, Immutability & Air-Gaps — Cybersecurity Fundamentals, GRC & Cryptography
- Three Brokers: Replication And Leaders — Apache Kafka & Event Streaming
- In-Sync Replicas And Minimum ISR — Apache Kafka & Event Streaming
- Requirements: The Constraints That Shape The System — Scalable System Design & Distributed Architecture
- Back Of The Envelope Estimation: Storage And Bandwidth — Scalable System Design & Distributed Architecture
- Replication, Failover And Leader Election — Scalable System Design & Distributed Architecture
- Partitioning, Sharding And Hot Keys — Scalable System Design & Distributed Architecture
- Eventual Consistency And What It Costs A Product — Scalable System Design & Distributed Architecture
References
- Replicating objects in Amazon S3
- Apache Kafka design: replication
- PostgreSQL documentation: High Availability, Load Balancing, and Replication
- AWS whitepaper: Disaster recovery options in the cloud
- MongoDB manual: Replication