spring-kafka

问题 How to deal with error handling in spring-kafka version 1.1.x ? As I see it was introduced in version 2.0... 回答1: In 1.x you can add an ErrorHandler implementation to the listener container (or container factory if you are using @KafkaListener annotations). The 2.0 feature was added to make it easier to configure if you want a different error handler for each listener when using annotations. 来源： https://stackoverflow.com/questions/43912292/spring-kafka-error-handling-v1-1-x

问题 I am manually starting Zookeeper, then Kafka server and finally the Kafka-Rest server with their respective properties file. Next, I am deploying my Spring Boot application on tomcat In the Tomcat log trace, I am getting the Error org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.KafkaException: Failed to construct kafka consumer and my application

问题 I'm using Spring Kafka 1.1.2-RELEASE with Spring Boot 1.5.0 RC and I have configured a custom value serialiser/deserialiser class extending org.springframework.kafka.support.serializer.JsonSerializer / org.springframework.kafka.support.serializer.JsonDeserializer . These classes do use a Jackson ObjectMapper which can be provided through the constructor. Is it somehow possible to inject the ObjectMapper from my Spring context? I have an ObjectMapper configured already which I would like to

问题 I'm using a compacted topic in kafka which I load into a HashMap at the application startup. Then I'm listening to a normal topic for messages, and processing them using the HashMap constructed from the compacted topic. How can I make sure the compacted topic is fully read and the HashMap fully initialized before starting to listen to the other topics ? (Same for RestControllers) 回答1: Implement SmartLifecycle and load the map in start() . Make sure the phase is earlier than any other object

问题 I use spring-kafka for springboot 2.0.4.RELEASE. And use KafkaListener for get message Now I want to reset the offset for my group But i do not how to get the consumer for the group @KafkaListener(id="test",topics={"test"},groupId="group",containerFactory="batchContainerFactory") public String listenTopic33(List<ConsumerRecord<Integer, String>> record, Acknowledgment ack){ // do something } @Autowired KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry; public void test() {

问题 Trying to figure out if I can write unit test for @KafkaListener using spring-kafka and spring-kafka-test. My Listener class. public class MyKafkaListener { @Autowired private MyMessageProcessor myMessageProcessor; @KafkaListener(topics = "${kafka.topic.01}", groupId = "SF.CLIENT", clientIdPrefix = "SF.01", containerFactory = "myMessageListenerContainerFactory") public void myMessageListener(MyMessage message) { myMessageProcessor.process(message); log.info("MyMessage processed"); }} My Test

问题 I am using spring kafka 2.2.7 and my consumer configuration code is following: @Slf4j @Configuration @EnableKafka public class KafkaConfiguration { @Bean ConcurrentKafkaListenerContainerFactory<String, Customer> kafkaListenerContainerFactory() { ConcurrentKafkaListenerContainerFactory<String, Customer> factory = new ConcurrentKafkaListenerContainerFactory<>(); factory.setConsumerFactory(consumerFactory()); // less than number of partition, will do infinite retry factory.setConcurrency(1);

问题 We would like to list all Kafka topics via spring-kafka to get results similar to the kafka command: bin/kafka-topics.sh --list --zookeeper localhost:2181 When running the getTopics() method in the service below, we get org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata Configuration: @EnableKafka @Configuration public class KafkaConfig { @Bean public ConsumerFactory<String, String> consumerFactory() { Map<String, Object> props = new HashMap<>();

问题 Using Kafka Broker: 1.0.1 spring-kafka: 2.1.6.RELEASE I'm using a batched consumer with the following settings: // Other settings are not shown.. props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, "100"); I use spring listener in the following way: @KafkaListener(topics = "${topics}", groupId = "${consumer.group.id}") public void receive(final List<String> data, @Header(KafkaHeaders.RECEIVED_PARTITION_ID) final List<Integer> partitions, @Header(KafkaHeaders.RECEIVED_TOPIC) Set<String> topics,

问题 Tried below SPEL expression, but it fails to work. Need help! @KafkaListener(topics = "#{Arrays.asList(${kafka.topic.helloworld}.split(',')).stream().map(p -> p+envSuffix).toArray(String[]::new)}") 回答1: First of all I see that ${kafka.topic.helloworld} has to be wrapped to the '' , just because property-placeholder works first and then SpEL will treat the result a an active variable. For example you have there, foo,bar,baz . How does it look in Java? Nothing more than wrong code. But when it