exactly-once semantics
A guarantee that each message's effects happen exactly once, even when senders retry or consumers crash.
In more detail
It needs the output and the record of progress to be saved atomically, usually with transactions, plus deduplication of retries. Effects in systems outside the transaction, such as sending an email, still need idempotent handling.
Example
Kafka builds it from idempotent producers, transactions and read_committed consumers.Related terms
- Kafka transaction — A group of writes, possibly across several partitions and topics, that Kafka makes visible all together on commit or not at all on abort.
- idempotent producer — A producer mode in which the broker detects and discards duplicate batches caused by retries, so each record is written once per partition.
- isolation.level — The consumer setting that decides whether records from open or aborted transactions are returned: read_uncommitted or read_committed.
- delivery semantics — The promise a messaging system makes about how many times each message is delivered: at most once, at least once or exactly once.
- at-least-once delivery — A delivery guarantee under which every message arrives, but some may arrive more than once.
- deduplication — Detecting and discarding repeated copies of the same message or request, usually by remembering identifiers already processed.
- idempotency — The property of an operation that gives the same result whether you run it once or many times, because a repeat run finds nothing left to do.
Taught in
- The Retry That Overtook — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Copies Of One Service — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Too Slow Between Two Polls — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Why A Log And Not A Queue — Apache Kafka & Event Streaming
- At Most Once, At Least Once — Apache Kafka & Event Streaming
- Duplicates Are The Default — Apache Kafka & Event Streaming
- Transactions And Exactly Once Semantics — Apache Kafka & Event Streaming
- Queues, Events And Asynchronous Work — Scalable System Design & Distributed Architecture
- Idempotency, And Exactly Once As A Fiction — Scalable System Design & Distributed Architecture
- Worked Design One: Ticket Booking, Correctness First — Scalable System Design & Distributed Architecture
- Worked Design Two: A Global Chat System — Scalable System Design & Distributed Architecture
References
- Apache Kafka design: using transactions
- Apache Kafka design: message delivery semantics
- Google Cloud Pub/Sub: Exactly-once delivery
- Apache Kafka documentation: Design