compression.type
The producer setting that compresses each batch with a codec such as gzip, snappy, lz4 or zstd before sending it.
In more detail
Compression works on whole batches, so bigger batches compress better. It saves network bandwidth and disk space at the cost of processor time on both the producer and the consumer.
Example
KafkaProducer(compression_type='gzip')Related terms
- gzip — The command that compresses a single file and replaces it with a smaller .gz version; gunzip reverses it.
- record batch — A group of records sent to the broker together in one request, instead of one request per record.
- throughput — How much work a system completes per unit of time, such as requests per second.
Taught in
- The Error Storm That Fixes Itself — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Retention Deletes Files, Not Records — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Brokers Down — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Batching, Linger And Throughput — Apache Kafka & Event Streaming