Enable JMX on Kafka Brokers

Question

I enabled JMX on Kafka brokers by adding

    `KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true 
     -Dcom.sun.management.jmxremote.authenticate=false  
     -Dcom.sun.management.jmxremote.ssl=false 
     -Djava.rmi.server.hostname=<server_IP> 
     -Djava.net.preferIPv4Stack=true`“. 

However, when I use ‘kafka.tools.JmxTool’ to get the metrics, it outputs Unix time stamps which are not the expected behavior.

    sh /usr/local/kafka/bin/kafka-run-class.sh kafka.tools.JmxTool --    object-name 'kafka.server:type=BrokerTopicMetrics,name=AllTopicsMessagesInPerSec' --jmx-url service:jmx:rmi:///jndi/rmi://<server_IP>:9111/jmxrmi

The above command prints out Unix time stamps. How can I fix this and have it print out the metrics?

Answer 1

Edit bin/kafka-run-class.sh and set KAFKA_JMX_OPTS variable

KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=your.kafka.broker.hostname -Djava.net.preferIPv4Stack=true"

Update bin/kafka-server-start.sh add the below line

export JMX_PORT=PORT

Answer 2

You must set 'JMX_PORT' variable, or add the following line to bin/kafka-server-start.sh.

export JMX_PORT=${JMX_PORT:-9999}

then you will be able to connect to Kafka JMX metrics. I use jconsole tool and 'localhost:9999' address.

Answer 3

Setting JMX_PORT inside bin/kafka-run-class.sh will clash with Zookeeper, if you are running Zookeeper on the same node. Best is to set JMX port individually inside corresponding server-start scripts:

1. Insert line “export JMX_PORT=${JMX_PORT:-9998}” before last line in $KAFKA_HOME/bin/zookeeper-server-start.sh file.
2. Restart the Zookeeper server.
3. Repeat steps 1 and 2 for all zookeeper nodes in the cluster.
4. Insert line “export JMX_PORT=${JMX_PORT:-9999}” before last line in $KAFKA_HOME/bin/kafka-server-start.sh file.
5. Restart the Kafka Broker.
6. Repeat steps 4 and 5 for all brokers in the cluster.

Answer 4

vim kafka_2.11-0.10.1.1/bin/kafka-run-class.sh

and then add the first two lines and comment as I have done for other lines, (Note : after doing this Kafka scripts cannot be used for client operations for listing topics.. for your client operations you need to use a separate scripts , download again in different locations and use)

export JMX_PORT=9096
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=<ipaddress> -Dcom.sun.management.jmxremote.port=$JMX_PORT -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"



# JMX settings
#if [ -z "$KAFKA_JMX_OPTS" ]; then
# KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false "

#fi

# JMX port to use
#if [  $JMX_PORT ]; then
#  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT "
#fi

Answer 5

If you're running via systemd:

1. edit /etc/systemd/system/multi-user.target.wants/kafka.service
2. in the "[service]" section add a line:
   • Environment=JMX_PORT=9989
3. reload: systemctl daemon-reload
4. restart: systemctl restart kafka
5. enjoy the beans: echo 'beans' | java -jar jmxterm-1.0-alpha-4-uber.jar -l localhost:9989 -n 2>&1

Answer 6

Use kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec

The AllTopics prefix was used in older verions. You can specify topic using kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=<topic-name>

src: http://grokbase.com/t/kafka/users/164ksnhff0/enable-jmx-on-kafka-brokers