internal topic
A topic the broker creates and uses for its own bookkeeping, such as __consumer_offsets, which stores committed offsets.
In more detail
Internal topics are hidden from rpk topic list by default. Every offset commit from a consumer group is written as a record to __consumer_offsets, which is why committed positions survive consumer restarts.
Related terms
- committed offset — The offset a consumer group has saved on the broker for a partition, marking the next record it should read after a restart.
- consumer group — A set of consumers sharing one group ID that divide a topic's partitions between them so each partition is read by only one member at a time.
- log compaction — A cleanup mode in which the broker keeps at least the latest record for every key and removes older records with the same key.
Taught in
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Twelve Lines Instead Of Four Hundred — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Sink That Died On Record Nine — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Upgrade That Could Not Start — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Night Before The Administrator Exam — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Starting A Broker And Reading Its Metadata — Apache Kafka & Event Streaming
- A Consumer In Python — Apache Kafka & Event Streaming