committed offset
The offset a consumer group has saved on the broker for a partition, marking the next record it should read after a restart.
In more detail
Reading records does not move it; only a commit does. Committing after processing gives at-least-once delivery, and committing before gives at-most-once. The committed offset is stored as a record in the internal __consumer_offsets topic.
Related terms
- enable.auto.commit — The consumer setting that, when true, commits offsets automatically on a background timer.
- manual offset commit — Committing a consumer's offsets by calling commit in your own code instead of relying on the automatic timer.
- consumer lag — How many records a consumer group has not yet committed on a partition: the log end offset minus the committed offset.
- internal topic — A topic the broker creates and uses for its own bookkeeping, such as __consumer_offsets, which stores committed offsets.
- at-least-once delivery — A delivery guarantee under which every message arrives, but some may arrive more than once.
- offset — The permanent position number of a record inside its partition, starting at zero and increasing by one for each record.
Taught in
- The Payment That Was Charged Twice — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Queue That Forgot — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Replay Everything Since Eleven — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Copies Of One Service — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Too Slow Between Two Polls — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Stuck On One Record — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Sink That Died On Record Nine — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Eleven Thousand Scans That Never Arrived — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Night Before The Administrator Exam — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Offsets: Position, Not Acknowledgement — Apache Kafka & Event Streaming
- A Consumer In Python — Apache Kafka & Event Streaming
- Committing Offsets: Auto, Manual And Lag — Apache Kafka & Event Streaming
- At Most Once, At Least Once — Apache Kafka & Event Streaming
- Watching Lag And Sizing Partitions — Apache Kafka & Event Streaming
- The Difference Between A Queue And A Log — Scalable System Design & Distributed Architecture