Getting JMX working under Tomcat 7 with SSL and a self-signed cert

Question

I'm trying to get JMX working under Tomcat 7.0.23 with SSL. The servers are located in AWS, which means all the hosts are NATed, and I need to use JmxRemoteLifecycleListener to explicitly set the two ports used by JMX. I've been doing a lot of reading on the subject but I just can't get all the pieces working together properly.

I can get JMX working fine without SSL. I have downloaded the version of catalina-jmx-remote.jar for my version of Tomcat and installed it in my tomcat/lib directory. My server.xml contains:

  <Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" 
        rmiRegistryPortPlatform="1099" rmiServerPortPlatform="1098" />

When I launch Tomcat with the following settings I can connect with an insecure session:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password 
-Dcom.sun.management.jmxremote.access.file=/path/to/jmxremote.access 
-Djava.rmi.server.hostname=<public IP of server> 
-Dcom.sun.management.jmxremote.ssl=false

However if I change these to the following then I'm unable to establish an SSL connection:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password 
-Dcom.sun.management.jmxremote.access.file=/path/to/jmxremote.access 
-Djava.rmi.server.hostname=<public IP of server> 
-Dcom.sun.management.jmxremote.ssl=true
-Dcom.sun.management.jmxremote.ssl.need.client.auth=false 
-Dcom.sun.management.jmxremote.authenticate=true
-Djavax.net.ssl.keyStore=/path/to/keystore.dat 
-Djavax.net.ssl.keyStorePassword=<password>
-Djavax.net.ssl.trustStore=/path/to/truststore.dat 
-Djavax.net.ssl.trustStorePassword=<password>

keystore.dat contains just a single certificate created via:

openssl x509 -outform der -in cert.pem -out cert.der
keytool -import -alias tomcat -keystore keystore.dat -file cert.der -storepass <password>

truststore.dat contains a full copy of the java cacerts plus the CA cert for my self-signed cert:

cp $JAVA_HOME/jre/lib/security/cacerts truststore.dat
keytool -storepasswd -storepass changeit -new <password> -keystore truststore.dat
keytool -import -trustcacerts -file mycacert.pem -alias myalias -keystore truststore.dat -storepass <password>

After launching Tomcat I've tried connecting via jconsole but it can't establish a connection. I tried to verify SSL using openssl but it looks like Tomcat isn't making use of the cert:

$ openssl s_client -connect <host>:1099
CONNECTED(00000003)
140735160957372:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 322 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

I've verified that my local keystore and truststore are set up properly by exporting the keys and verifying the cert chain (combined.pem is all the CA certs from truststore.dat and cert.pem is my cert from keystore.dat):

$ openssl verify -verbose -purpose sslserver -CAfile combined.pem cert.pem
cert.pem: OK

So now I'm at a complete loss. The cert and CA cert look correct. Unencrypted JMX connections work. But I can't seem to get the connection to use SSL. What am I missing here?

I don't know if this is just a red herring or not, but I don't see any way to specify what cert in the keyStore is used by JMX. Some of what I read implies that it just uses a cert with the alias "tomcat". Is that correct?

Answer 1

You are starting the RMI Registry on port 1099. In order for the RMI Registry to utilize SSL you need to pass an additional -Dcom.sun.management.jmxremote.registry.ssl=true argument.

Edit: Do you use jconsole -J-Djavax.net.ssl.trustStore=truststore -J-Djavax.net.ssl.trustStorePassword=trustword -Djavax.net.ssl.keyStore=keystore -Djavax.net.ssl.keyStorePassword=password to start JConsole? It needs to know where the stores are and the corresponding passwords.

For VisualVM you can install VisualVM-Security plugin which will add a new tab to the Options dialogue allowing you customize the SSL related options in UI.

Answer 2

I wanted tomcat getting to work with jmx and ssl and followed Bruce setup. In case someone runs int the same problem: using the openssl and keytool commands form Bruce I was getting at client ssl errors:

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Not sure why but the figerprint on one side was 5 chars longer at the beginning.

I recreated the keystore using:

keytool -genkeypair -alias tomcat -keyalg RSA -keystore keystore.jks -dname cn=test,ou=test,dc=example,dc=com

In tomcat sever.xml I added the listener as Bruce suggested.

Because client auth is set to false I do not add truststore in tomcat jmx configuration. Instead I added registry.ssl:

[...]
-Djavax.net.ssl.keyStorePassword=your_keystore_pass
-Dcom.sun.management.jmxremote.registry.ssl=true

Then on client side as J.B. suggested I download ssl plugin for visualvm and criss-cross the keystore resulting in:

visualvm -J-Djavax.net.ssl.trustStore=keystore.jks -J-Djavax.net.ssl.trustStorePassword=your_keystore_pass

When adding authentication make sure that your jmx access file looks like:

<user_name>    readwrite

and your password file looks like:

<user_name>    <your_password>

This solved my ssl jmx setup for testing with visualm vm.

Edit:

Had some issues with having the registry also over SSL, set by:

-Dcom.sun.management.jmxremote.registry.ssl=true

leading at client to throw:

Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint

Adding client auth authentication:

-Dcom.sun.management.jmxremote.ssl.need.client.auth=true
-Djavax.net.ssl.trustStore=truststore.jks
-Djavax.net.ssl.trustStorePassword=your_trust_store_pass

solved both jmx and its registry to use ssl.