kafka consumer to dynamically detect topics added

Question

I\'m using KafkaConsumer to consume messages from Kafka server (topics)..

• It works fine for topics created before starting Consumer code...

But

Answer 1

You can hook into Zookeeper. Check out the sample code. In essence, you will create a watcher on the Zookeeper node /brokers/topics. When new children are added here, it's a new Topic being added, and your watcher will get triggered.

Note that the difference between this and the other answer is that this one is a trigger where the other is a polling -- this one will be as close to real-time as possible, the other will be within whatever your polling interval is at best.