consumer

I have a situation where different threads populate a queue (producers) and one consumer retrieve element from this queue. My problem is that when one of these elements are retrieved from the queue some is missed (missing signal?). The producers code is: class Producer implements Runnable { private Consumer consumer; Producer(Consumer consumer) { this.consumer = consumer; } @Override public void run() { consumer.send("message"); } } and they are created and run with: ExecutorService executor = Executors.newSingleThreadExecutor(); for (int i = 0; i < 20; i++) { executor.execute(new Producer

I have a two-nodes kafka cluster (EC2 instances) where each node is used as a separate broker. When I run a producer on the leader instance with the following command: kafka-console-producer.sh --broker-list localhost:9092 --topic test I get the following errors. test message [2017-01-09 13:22:39,483] WARN Error while fetching metadata with correlation id 0 : {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-01-09 13:22:39,562] WARN Error while fetching metadata with correlation id 1 : {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)

While using the disruptor, there may be a consumer(s) that is lagging behind, and because of that slow consumer, the whole application is affected. Keeping in mind that every producer(Publisher) and consumer(EventProcessor) is running on a single thread each, what can be the solution to the slow consumer problem? Can we use multiple threads on a single consumer? If not, what is a better alternative? Generally speaking use a WorkerPool to allow multiple pooled worker threads to work on a single consumer, which is good if you have tasks that are independent and of a potentially variable duration

I'm using latest version of kafka(kafka_2.12-1.0.0.tgz). I have setup simple cluster with 3 brokers(just changed broker.id=1 and listeners=PLAINTEXT://:9092 in properties file for each instance).After cluster is up I created topic with the following command ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 13 --topic demo then start kafka consumer and producers with the following commands ./kafka-console-producer.sh --topic demo --broker-list localhost:9094,localhost:9093,localhost:9092 ./kafka-console-consumer.sh --group test --bootstrap-server

I am introducing with kafka and I want to know how to specify partition when I consume messages from topic. I have found several picture like this: It means that 1 consumer can consume messages from several partitions but 1 partition can be read by single consumer(within consumer group) Also I have read several examples for consumer and it looks like this: Properties props = new Properties(); props.put("bootstrap.servers", "localhost:9092"); props.put("group.id", "consumer-tutorial"); props.put("key.deserializer", StringDeserializer.class.getName()); props.put("value.deserializer",

QueueingConsumer consumer = new QueueingConsumer(channel); System.out.println(consumer.getConsumerTag()); channel.basicConsume("queue1", consumer); channel.basicConsume("queue3", consumer); Is it possible to stop consuming the messages from the queue "queue3" alone dynamically? Yes you can, using channel.basicCancel(consumerTag); EDIT For example: String tag3 = channel.basicConsume("queue3", consumer); channel.basicCancel(tag3) Here you can find a code that unsubscribe a consumer after 5 seconds: String tag1 = channel.basicConsume(myQueue, autoAck, consumer); String tag2 = channel.basicConsume

I have a class structure like this: public class Foo { private FooB foob; public Optional<FooB> getFoob() { return Optional.ofNullable(foob); } } public class FooB { private int valA; public int getValA() { return valA; } } My objective is to call the get method for fooB and then check to see if it's present. If it is present then return the valA property, if it doesn't then just return null. So something like this: Integer valA = foo.getFoob().ifPresent(getValA()).orElse(null); Of course this isn't proper Java 8 optional syntax but that's my "psuedo code". Is there any way to achieve this in