I'm trying to monitor the lag of a consumer group in Kafka 0.10.
Our consumers are keeping track of their offsets in Kafka rather than ZooKeper. This means I can get the figures using the following:
bin/kafka-consumer-groups.sh --bootstrap-server <broker> --describe --group <group-name>
This works fine. However, my broker already makes use of the Prometheus JMX exporter for collecting a number of stats. I've connected JConsole to the brokers but can't see the same data being reported in JMX as reported by kafka-consumer-groups.sh.
Is there anyway to get this information from Kafka with JMX without needing any additional tools?
You could retrieve the atrributes {topic}-{partition}.records-lag of metric kafka.consumer:type=consumer-fetch-manager-metrics,client-id={client-id} for all partitions. That should be equivalent to the output of consumer-groups.sh
To expose JMX metrics from consumer simply add
-Dcom.sun.management.jmxremote.port=9090
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
parameters to your java based consumer application, then you can access it using jconsole tool and localhost:9090 url. There you can find kafka.consumer:type=consumer-fetch-manager-metrics bean.
来源：https://stackoverflow.com/questions/46613797/kafka-consumer-lag-through-jmx