Kafka consumer list

Question

I need to find out a way to ask Kafka for a list of topics. I know I can do that using the kafka-topics.sh script included in the bin\\ directory.

Answer 1

Use kafka-consumer-groups.sh

For example

bin/kafka-consumer-groups.sh  --list --bootstrap-server localhost:9092

bin/kafka-consumer-groups.sh --describe --group mygroup --bootstrap-server localhost:9092

Answer 2

All the consumers per topic

(Replace --zookeeper with --bootstrap-server to get groups stored by newer Kafka clients)

Get all consumers-per-topic as a table of topictabconsumer:

for t in `kafka-consumer-groups.sh --zookeeper <HOST>:2181 --list 2>/dev/null`; do
    echo $t | xargs -I {} sh -c "kafka-consumer-groups.sh --zookeeper <HOST>:2181 --describe --group {} 2>/dev/null | grep ^{} | awk '{print \$2\"\t\"\$1}' "
done > topic-consumer.txt

Make this pairs unique:

cat topic-consumer.txt | sort -u > topic-consumer-u.txt

Get the desired one:

less topic-consumer-u.txt | grep -i <TOPIC>