kafka-consumer-api

I am trying to use Kafka Connect JDBC sink connector to insert data into Oracle but it is throwing an error . I have tried with all the possible configurations of the schema. Below is the examples . Please suggest if I am missing anything below are my configurations files and errors. Case 1- First Configuration internal.value.converter.schemas.enable=false . so I am getting the [2017-08-28 16:16:26,119] INFO Sink task WorkerSinkTask{id=oracle_sink-0} finished initialization and start (org.apache.kafka.connect.runtime.WorkerSinkTask:233) [2017-08-28 16:16:26,606] INFO Discovered coordinator dfw

We start a Kafka consumer, listening on a topic which may not yet be created (topic auto creation is enabled though). Not long thereafter a producer is publishing messages on that topic. However, it takes some time for the consumer to notice this: 5 minutes to be exact. At this point the consumer revokes its partitions and rejoins the consumer group. Kafka re-stabilizes the group. Looking at the time-stamps of the consumer vs. kafka logs, this process is initiated at the consumer side. I suppose this is expected behavior but I would like to understand this. Is this actually a re-balancing

I am investigating Kafka 9 as a hobby project and completed a few "Hello World" type examples. I have got to thinking about Real World Kafka applications based on request response messaging in general and more specifically how to link a Kafka request message to its response message. I was thinking along the lines of using a generated UUID as the request message key and employ this request UUID as the associated response message key. Much the same type of mechanism that WebSphere MQ has message correlation id. My end 2 end process would be. 1). Kafka client generates a random UUID and sends a

I am new to Kafka. I noticed in Consumer configuration that has two ids. one is group.id ( mandatory ) and second one is consumer.id ( non Mandatory ). Please tell why 2 Ids and difference. Consumers groups is a Kafka abstraction that enables supporting both point-to-point and publish/subscribe messaging. A consumer can join a consumer group (let us say group_1 ) by setting its group.id to group_1 . Consumer groups is also a way of supporting parallel consumption of the data i.e. different consumers of the same consumer group consume data in parallel from different partitions. In addition to

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\ directory. Once I have this list, I need all the consumers per topic. I could not find a script in that directory, nor a class in the kafka-consumer-api library that allows me to do it. The reason behind this is that I need to figure out the difference between the topic's offset and the consumers' offsets. Is there a way to achieve this? Or do I need to implement this functionality in each of my consumers? Use kafka-consumer-groups.sh For example bin/kafka