consumer

问题 So I am learning Kafka currently and have attempted to duplicate the examples provided from Apache here. This is example code for the consumer and I have written it in java just as shown. When I attempt to execute the file however I run into some issues. I am able to get the file to compile but it will not run properly. I am executing the program with the following line without the quotations, "java TestConsumer localhost:2181 group1 test 4" This passes the 4 arguments necessary in the

问题 I am using pactNet to test an API which should return an array of a flexible length. If i call "myApi/items/" it should return a list of items where the consumer does not know the exact size of. So the answer should look like this: [ { "id": "1", "description": "foo" }, { "id": "2", "description": "foo2" }, { "id": "3", "description": "foo3" } ] or this: [ { "id": "4", "description": "foo4" }, { "id": "2", "description": "foo2" } ] How do I create the contract for this interaction? In the

问题 I need data from kafka brokers,but for fast access I am using multiple consumers with same group id known as consumer groups.But after reading by each consumer,how can we combine data from multiple consumers? Is there any logic? 回答1: By design, different consumers in the same consumer group process data independently from each other. (This behavior is what allows applications to scale well.) But after reading by each consumer,how can we combine data from multiple consumers? Is there any logic

问题 I am trying to make use of mirror maker to replicate __consumre_offsets topic along with other topics. It is giving error as mentioned below. [2018-10-24 16:16:03,802] ERROR Error when sending message to topic __consumer_offsets with key: 16 bytes, value: 445 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.InvalidTopicException: The request attempted to perform an operation on an invalid topic. [2018-10-24 16:16:03,828] ERROR

问题 I am testing the Kafka High Level Consumer using the ConsumerGroupExample code from the Kafka site. I would like to retrieve all the existing messages on the topic called "test" that I have in the Kafka server config. Looking at other blogs, auto.offset.reset should be set to "smallest" to be able to get all messages: private static ConsumerConfig createConsumerConfig(String a_zookeeper, String a_groupId) { Properties props = new Properties(); props.put("zookeeper.connect", a_zookeeper);

问题 I read you can have multiple consumer apps per kinesis stream. http://docs.aws.amazon.com/kinesis/latest/dev/developing-consumers-with-kcl.html however, I heard you can only have on consumer per shard. Is this true? I don't find any documentation to support this, and can't imagine how that could be if multiple consumers are reading from the same stream. Certainly, it doesn't mean the producer needs to repeat content in different shards for different consumers. 回答1: Kinesis Client Library

问题 I have a class structure like this: public class Foo { private FooB foob; public Optional<FooB> getFoob() { return Optional.ofNullable(foob); } } public class FooB { private int valA; public int getValA() { return valA; } } My objective is to call the get method for fooB and then check to see if it's present. If it is present then return the valA property, if it doesn't then just return null. So something like this: Integer valA = foo.getFoob().ifPresent(getValA()).orElse(null); Of course

问题 I’m writing a program that implements the Producer Consumer problem in Java using multithreading concepts. Below are few details how I’m supposed to do it: 1) The main thread should create a buffer with capacity specified as a command line argument. The number of producer and consumer threads are also specified as command line arguments. I’m supposed to assign a unique number to each producer and consumer thread. How do I assign a unique number to producer and consumer threads? 2) The

问题 Please copy the program below and try running in your IDE. It's a simple Produce Consumer implementation - it runs fine when I use one Producer and one Consumer thread but fails when using 2 each. Please let me know the reason why this program hangs or is there anything else wrong with it. import java.util.LinkedList; import java.util.Queue; public class PCQueue { private volatile Queue<Product> productQueue = new LinkedList<Product>(); public static void main(String[] args) { PCQueue pc =

问题 I need to use something to coordinate my system with several consumers/producers each running on different machines with different operating systems. I have been researching on using MySql to do this, but it seems ridiculously difficult. My requirements are simple: I want to be able to add or remove consumers/producers at any time and thus they should not depend on each other at all. Naturally a database would separate the two nicely. I have been looking at Q4M message queuing plugin for