kafka monitoring using JMX when running kafka in docker in boot2docker

Question

I am trying to get JMX monitoring working to monitor a test kafka instance.

I have kafka (ches/kafka) running in docker via boot2docker but I can not get JMX monitoring configured properly. I have done a bunch of troubleshooting and I know the kafka instance is running properly (consumers and producers work). The issue arises when I try simple JMX tools (jconsole and jvisualvm) and both can not connect (insecure connection error, connection failed).

Configuration items of note: I connect to 192.168.59.103 (virtualbox ip found when running 'boot2docker ip') and the ches/kafka docker/kafka instance uses the port 7203 as the JMX_PORT (confirmed in the kafka startup logs). Using jconsole, I connect to 192.168.59.103:7203 and that is when the errors occur.

Any help is appreciated.

Answer 1

For completeness, here is the solution that works:

I ran ches/kafka docker image as follows -- note that the JMX_PORT (7203) is now published appropriately:

$ docker run --hostname localhost --name kafka --publish 9092:9092 --publish 7203:7203 --env EXPOSED_HOST=192.168.59.103 --env ZOOKEEPER_IP=192.168.59.103 ches/kafka

Also, the following environment is set in the kafka-run-class.sh (.bat for windows)

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

But I needed to add one additional item (thanks to one of the commenters for pointing this out):

-Dcom.sun.management.jmxremote.rmi.port=7203

Now, to run the ches/docker image in boot2docker you just need to set one of the recognized environment variables (KAFKA_JMX_OPTS or KAKFA_OPTS) to add the additional item and it now works.

Thanks for the help!

Answer 2

There's no reason the kafka docker port would bind to the same port in the boot2docker VM except if you specify it.

Try running it with -p 7203:7203 to force the 1:1 forwarding of the port.