poll loop
The loop in which a consumer repeatedly asks the broker for new records and processes them.
In more detail
Each call to poll fetches the next records after the consumer's position. In kafka-python, a for loop over the consumer calls poll for you. If processing between polls takes longer than max.poll.interval.ms, the group assumes the consumer is stuck and rebalances.
Related terms
- KafkaConsumer — The kafka-python class that reads records from Kafka topics, joins consumer groups and commits offsets.
- max.poll.interval.ms — The consumer setting for the longest gap allowed between two poll calls before the consumer is treated as stuck and removed from the group.
- consumer_timeout_ms — A kafka-python consumer setting that ends the for loop over the consumer after this many milliseconds without a new record.
Taught in
- Where A Record Lands — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Copies Of One Service — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Too Slow Between Two Polls — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Twenty Seconds Of Nothing — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Green Tests, Red Production — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Alert That Came An Hour Late — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Stuck On One Record — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Eleven Thousand Scans That Never Arrived — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Seventy One Percent — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Consumer In Python — Apache Kafka & Event Streaming
- Rebalancing And Its Cost — Apache Kafka & Event Streaming
- At Most Once, At Least Once — Apache Kafka & Event Streaming