Kafka consumer list

Question

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\\ directory.

Answer 1

High level consumers are registered into Zookeeper, so you can fetch a list from ZK, similarly to the way kafka-topics.sh fetches the list of topics. I don't think there's a way to collect all consumers; any application sending in a few consume requests is actually a "consumer", and you cannot tell whether they are done already.

On the consumer side, there's a JMX metric exposed to monitor the lag. Also, there is Burrow for lag monitoring.