spring-kafka

问题 I have spring boot application that had a consumer consumes from topic in one cluster and produces to another topic in different cluster. Now I'm trying to write integration test case using spring embedded Kafka but having an issue KafkaTemplate could not be registered. A bean with that name has already been defined in class path resource Consumer Class @Service public class KafkaConsumerService { @Autowired private KafkaProducerService kafkaProducerService; @KafkaListener(topics = "${kafka

问题 I am trying to write kafka consumer using spring-kafka version 2.3.0.M2 library. To handle run time errors I am using SeekToCurrentErrorHandler.class with DeadLetterPublishingRecoverer as my recoverer. This works fine only when my consumer code throws exception, but fails when unable to deserialize the message. I tried implementing ErrorHandler myself and I was successful but with this approach I myself end up writing DLT code to handle error messages which I do not want to do. Below are my

问题 Question from Twitter: Just trying to find out a simple example with spring-kafka 2.1.7 that works with a KafkaListener and AckMode.MANUAL_IMMEDIATE , to retry last failed message. https://twitter.com/tolbier/status/1028936942447149056 回答1: it's generally better to ask such questions on Stack Overflow (tagged with spring-kafka. There are two ways: Add a RetryTemplate to the listener container factory - the retries will be performed in memory and you can set backoff properties. Add a

问题 Hy, I'm using spring-kafka 1.3.0.RELEASE to create a transactional producer. When the bootstrap server is down, the DefaultKafkaProducerFactory waits endlessly until the bootstrap server is up. What am I doing wrong ? Can I set a timeout and/or other similar properties ? This is an example of my code to reproduce the scenario: public static void main(String[] args) { final DefaultKafkaProducerFactory<Object, Object> producerFactory = new DefaultKafkaProducerFactory<>(producerConfigs());

问题 We have an issue with one of our Kafka topics which is consumed by the DefaultKafkaConsumerFactory & ConcurrentMessageListenerContainer combination described here with a JsonDeserializer used by the Factory. Unfortunately someone got a little enthusiastic and published some invalid messages onto the topic. It appears that spring-kafka silently fails to process past the first of these messages. Is it possible to have spring-kafka log an error and continue? Looking at the error messages which

问题 Can I please check with the community what is the best way to listen to multiple topics, with each topic containing a message of a different class? I've been playing around with Spring Kafka for the past couple of days. My thought process so far: Because you need to pass your deserializer into DefaultKafkaConsumerFactory when initializing a KafkaListenerContainerFactory. This seems to indicate that if I need multiple containers each deserializing a message of a different type, I will not be

问题 Can I please check with the community what is the best way to listen to multiple topics, with each topic containing a message of a different class? I've been playing around with Spring Kafka for the past couple of days. My thought process so far: Because you need to pass your deserializer into DefaultKafkaConsumerFactory when initializing a KafkaListenerContainerFactory. This seems to indicate that if I need multiple containers each deserializing a message of a different type, I will not be

问题 I have implemented a Kafka consumer with KafkaHandler . My consumer is supposed to consume events, then send a REST request to some other service for each event. I want to retry only if that REST service is down. Otherwise, I can ignore the failed event. My container factory is configured as below: @Bean public ConcurrentKafkaListenerContainerFactory<String, MyCustomEvent> kafkaListenerContainerFactory() { ConcurrentKafkaListenerContainerFactory<String, MyCustomEvent> factory = new

问题 I am doing a POC with Spring Boot & Kafka for a transactional project and I have the following doubt: Scenario: One microservices MSPUB1 receives Requests from the customer. That requests publish a message on topic TRANSACTION_TOPIC1 on Kafka but that Microservice could receive multiple requests in parallel. The Microservice listens the topic TRANSACTION_RESULT1 to check that the transaction finished. In the other side of the Streaming Platform, another Microservice MSSUB1 is listening the

问题 I am doing a POC with Spring Boot & Kafka for a transactional project and I have the following doubt: Scenario: One microservices MSPUB1 receives Requests from the customer. That requests publish a message on topic TRANSACTION_TOPIC1 on Kafka but that Microservice could receive multiple requests in parallel. The Microservice listens the topic TRANSACTION_RESULT1 to check that the transaction finished. In the other side of the Streaming Platform, another Microservice MSSUB1 is listening the