offset
The permanent position number of a record inside its partition, starting at zero and increasing by one for each record.
In more detail
An offset is an address, not a receipt: reading a record does not change or remove it, and any consumer can start reading from any offset. The partition and offset together identify exactly one record. Consumer groups store their progress separately as a committed offset.
Example
rpk topic consume events -o 3 --num 2Related terms
- partition — One of several independent ordered logs a message stream is split into, with each message routed to one partition by its key.
- committed offset — The offset a consumer group has saved on the broker for a partition, marking the next record it should read after a restart.
- high watermark — The offset one past the last record that every in-sync replica has copied, which is as far as consumers are allowed to read.
- replay — Delivering or returning something again: a stored result for a repeated request, or past events for a client that missed them.
- consumer lag — How many records a consumer group has not yet committed on a partition: the log end offset minus the committed offset.
Taught in
- The Payment That Was Charged Twice — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Queue That Forgot — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Where A Record Lands — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Acknowledged Is Not Always Safe — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Retention Deletes Files, Not Records — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Replay Everything Since Eleven — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Retry That Overtook — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Copies Of One Service — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Too Slow Between Two Polls — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Green Tests, Red Production — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Stuck On One Record — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Twelve Lines Instead Of Four Hundred — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Sink That Died On Record Nine — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty One Under-Replicated Partitions — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Eleven Thousand Scans That Never Arrived — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty Minutes To Start — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Data Centre That Went Dark — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Seventy One Percent — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Night Before The Administrator Exam — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Why A Log And Not A Queue — Apache Kafka & Event Streaming
- Offsets: Position, Not Acknowledgement — Apache Kafka & Event Streaming
- A Producer In Python — Apache Kafka & Event Streaming
- A Consumer In Python — Apache Kafka & Event Streaming
- The Difference Between A Queue And A Log — Scalable System Design & Distributed Architecture
References
- Apache Kafka design: consumer position
- Apache Kafka: main concepts and terminology
- Apache Kafka documentation