consumer group
A set of consumers sharing one group ID that divide a topic's partitions between them so each partition is read by only one member at a time.
In more detail
Consumer groups give Kafka queue-like work sharing on top of the log, and each group keeps its own committed offsets. Two different groups read the same records independently. When members join or leave, the group rebalances and redistributes partitions.
Related terms
- group.id — The consumer setting that names the consumer group, under which the broker stores committed offsets.
- committed offset — The offset a consumer group has saved on the broker for a partition, marking the next record it should read after a restart.
- rebalance — The process in which a consumer group redistributes partitions among its members after one joins, leaves or is judged dead.
- partition assignment — The decision about which consumer in a group reads which partitions.
- consumer lag — How many records a consumer group has not yet committed on a partition: the log end offset minus the committed offset.
- consumer — A program that receives and processes messages or events, or more broadly any client that uses an API.
- partition — One slice of a larger dataset or message stream, owned and processed independently of the others.
Taught in
- The Payment That Was Charged Twice — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Ninety Minutes, No Question Count — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Queue That Forgot — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Where A Record Lands — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Replay Everything Since Eleven — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Copies Of One Service — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Too Slow Between Two Polls — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Twenty Seconds Of Nothing — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Blank Names After Every Deploy — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Hours Of Scans At Once — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Alert That Came An Hour Late — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Stuck On One Record — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Twelve Lines Instead Of Four Hundred — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Sink That Died On Record Nine — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Allowed To Read, Refused Anyway — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Forty One Under-Replicated Partitions — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Eleven Thousand Scans That Never Arrived — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Data Centre That Went Dark — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Full Disks On Black Friday — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Night Before The Administrator Exam — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Why A Log And Not A Queue — Apache Kafka & Event Streaming
- A Consumer In Python — Apache Kafka & Event Streaming
- Consumer Groups And Partition Assignment — Apache Kafka & Event Streaming
- Materialised Views From A Stream — Apache Kafka & Event Streaming
- Watching Lag And Sizing Partitions — Apache Kafka & Event Streaming
- Queues, Events And Asynchronous Work — Scalable System Design & Distributed Architecture
References
- Apache Kafka: main concepts and terminology
- Apache Kafka operations: managing consumer groups
- Apache Kafka documentation