Improving performance of Kafka Producer

Question

We\'re running on apache kafka 0.10.0.x and spring 3.x and cannot use spring kafka as it is supported with spring framework version 4.x.

Therefore, we are using the

Answer 1

In the below conditions you need to configure batch.size, linger.ms & compression.type properties in your kafka prodocer to increase the performance.

1) If records are arriving faster than the kafka producer can send.

2) If you have huge amount of data in the your respective Topic, its really burden to your kafka producer.

3) if you have a bottlenecks

batch.size = 16_384 * 4
 linger.ms 200
compression.type = "snappy"

props.put(ProducerConfig.BATCH_SIZE_CONFIG, 16_384 * 4);
    // Send with little bit buffering
    props.put(ProducerConfig.LINGER_MS_CONFIG, 200);    
  //Use Snappy compression for batch compression.
    props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "snappy");

kafka Dzone

Performance tunning

Kafka Perforamnce tunning