DefaultKafkaProducerFactory with transactionIdPrefix endless waits when bootstrap server is down

Question

Hy,

I'm using spring-kafka 1.3.0.RELEASE to create a transactional producer. When the bootstrap server is down, the DefaultKafkaProducerFactory waits endlessly until the bootstrap server is up.

What am I doing wrong ? Can I set a timeout and/or other similar properties ?

This is an example of my code to reproduce the scenario:

public static void main(String[] args) {

    final DefaultKafkaProducerFactory<Object, Object> producerFactory = new DefaultKafkaProducerFactory<>(producerConfigs());    

    producerFactory.setTransactionIdPrefix("transactionIdPrefix");

    final Producer<Object, Object> producer = producerFactory.createProducer();       

    System.out.println("Created producer:" + producer);  
}

private static Map<String, Object> producerConfigs() {

    final Map<String, Object> props = new HashMap<>();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.56.1:9092");
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");

    props.put(ProducerConfig.RETRIES_CONFIG, 1);
    props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, 1);

    props.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, 1000);    
    props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, 1000);
    props.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 1000);

   return props;
}

Answer 1

It's caused by the factory calling initTransactions() on the producer after creating it, for example if there are not enough brokers to support the transaction log replication factor.

I don't know why the timeouts don't apply to that operation.

We could probably change the factory to defer the initTransactions() until the first beginTransaction() - but that will just push the problem downstream.

I tested with the kafka 1.0.0 client (which can be used with 1.3.1 or higher - currently 1.3.2) and it's still a problem there. I think it should honor the TRANSACTION_TIMEOUT_CONFIG but it appears not to.

I suggest opening an issue on the Kafka JIRA.