kafka-partition

How Kafka guarantee the messages order while we increase the partitions in runtime?

谁都会走 提交于 2021-01-27 16:28:25
问题 I am new to kafka and when I read the Kafka doc, I realize that messages provided with the same key will be mapped to the same partition to guarantee the order. This totally makes sense. However, I'd like to know if we increase the number of topic partitions in runtime, will the new messages with the same key be hashed to the same partition (old one) as before? If so, what if all messages are provided with keys, then none of them will be mapped to new partition? This doesn't make sense to me.

Is it possible to consume kafka messages using key and partition?

六月ゝ 毕业季﹏ 提交于 2020-01-16 01:20:56
问题 I am using kafka_2.12 version 2.3.0 where I am publishing data into kafka topic using partition and key. I need to find a way using which I can consume a particular message from topic using key and partition combination. That way I won't have to consume all the messages and iterate for the correct one. Right now I am only able to do this KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(props) consumer.subscribe(Collections.singletonList("topic")) ConsumerRecords

Is it possible to consume kafka messages using key and partition?

一世执手 提交于 2020-01-16 01:20:31
问题 I am using kafka_2.12 version 2.3.0 where I am publishing data into kafka topic using partition and key. I need to find a way using which I can consume a particular message from topic using key and partition combination. That way I won't have to consume all the messages and iterate for the correct one. Right now I am only able to do this KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(props) consumer.subscribe(Collections.singletonList("topic")) ConsumerRecords