spring-kafka

问题 I'm trying to setup a project that uses Spring Boot 1.5.10 and Spring Integration. It seems that Spring Boot 1.5.x uses the old version of Spring Integration, which is 4.3.14. The project also requires spring-integration-kafka which is based on Spring Integration 5.0.1. So, I'm ending up in a bit of dependencies hell. Is there a way for Spring Boot 1.5.x to use the latest Spring Integration? 回答1: I have Spring Boot 1.5.9 and I upgraded to Spring Integration IP 5.0.2. I'm guessing this is why

问题 I'm trying to setup a project that uses Spring Boot 1.5.10 and Spring Integration. It seems that Spring Boot 1.5.x uses the old version of Spring Integration, which is 4.3.14. The project also requires spring-integration-kafka which is based on Spring Integration 5.0.1. So, I'm ending up in a bit of dependencies hell. Is there a way for Spring Boot 1.5.x to use the latest Spring Integration? 回答1: I have Spring Boot 1.5.9 and I upgraded to Spring Integration IP 5.0.2. I'm guessing this is why

问题 We are using Kafka as a Strictly Ordered Queue and hence a single topic/single partition/single consumer group combo is in use. I should be able to use multiple partition later in future. My consumer is spring-boot app listener, that produces and consumes from the same topic(s). So the consumer group is fixed and there is always a single consumer. Kafka version 0.10.1.1 In such scenario the Log file for topic-0 and a few __consumer_offsets_XX grows. In fact __consumer_offsets_XX grows very

问题 Sorry for the question being too generic, but someone has some tutorial or guide on how to perform producer and consumer testing with kafka embedded. I've tried several, but there are several versions of dependencies and none actually works =/ I'm using spring cloud stream kafka. 回答1: We generally recommend using the Test Binder in tests but if you want to use an embedded kafka server, it can be done... Add this to your POM... <dependency> <groupId>org.springframework.kafka</groupId>

问题 I am using Kafka v0.10.1.1 with Spring-boot . I am trying to produce a message in a Kafka topic mobile-user using the below producer code: Topic mobile-user have 5 partitions and 2 replication factor . I have attached my Kafka settings at the end of the question . package com.service; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.kafka.core.KafkaTemplate; import org.springframework.kafka

问题 I'm trying to implement an asynchronous REST method of sending a message to Kafka in Spring MVC. Everything works, but when the server is unavailable, the onFailure event is processed for a long time. How to limit the response time in ListenableFuture for example to three seconds. Here's my code: @Autowired KafkaTemplate<String, String> kafkaTemplate; @Value("${spring.kafka.topic}") String topic; @RequestMapping("/test") DeferredResult<ResponseEntity<?>> test( @RequestParam(value = "message")

问题 I am currently implementing a micro service, which reads data from Apache Kafka topic. I am using "spring-boot, version: 1.5.6.RELEASE" for the micro service and "spring-kafka, version: 1.2.2.RELEASE" for the listener in the same micro service. This is my kafka configuration: @Bean public Map<String, Object> consumerConfigs() { return new HashMap<String, Object>() {{ put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, servers); put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer

问题 I'm using the annotation @KafkaListener to consume topics on my application. My issue is that if I create a new topic in kafka but my consumer is already running, it seems the consumer will not pick up the new topic, even if it matches with the topicPattern I'm using. Is there a way to "refresh" the subscribed topics periodically, so that new topics are picked up and rebalanced upon my running consumers? I'm using Spring Kafka 1.2.2 with Kafka 0.10.2.0. Regards 回答1: You can't dynamically add

问题 Can anyone provide me a small example in spring boot kafka where we can consume multiple topics in one single listener class. 回答1: application.yml my: kafka: conf: groupId: myId topics: topic1,topicN you listener: @KafkaListener(groupId = "${my.kafka.conf.groupId}", topics = "#{'${my.kafka.conf.topics}'.split(',')}") public void storeTopicsDataToMongo( @Header(KafkaHeaders.RECEIVED_TOPIC) String topic, @Header(required = false, name= KafkaHeaders.RECEIVED_MESSAGE_KEY) String key, @Payload

问题 I have a spring boot application in which I have single Kafka Consumer. I am using a DefaultKafkaConsumerFactory with default Consumer Configurations. I have a ConcurrentListenerContainerFactory with concurrency set to 1, and I have a method annotated with @KafkaListener. I am listening to a topic with 3 partitions and I have 3 of such consumers deployed each in different applications. Hence, each consumer is listening to one partition. Lets say poll on the consumer is called under the hood