idempotent producer

A producer mode in which the broker detects and discards duplicate batches caused by retries, so each record is written once per partition.

In more detail

The broker gives the producer a producer ID, and the producer stamps every batch with a sequence number. A retried batch with a sequence the broker has already written is acknowledged but not stored again. It protects only against retries from one producer session, not against your code sending the same data twice.

Example

KafkaProducer(acks='all', enable_idempotence=True, max_in_flight_requests_per_connection=1)

Related terms

Taught in

References

Certification courses