SSLHandshakeException: Received fatal alert: handshake_failure after Java 6 -> 8 upgrade

Question

We\'ve recently updated a project from Java 6 to Java 8 and now we\'ve hit a brick wall regarding SSL handshake.

The service layer uses a client to request and recei

Answer 1

Alright, so we've got it working now. I'll post the answer here in case someone might need it some day.

We have tried quite a few things, so I'm not exactly sure what actually needed to be done in order for it to work, but here are some of the things we changed in the process.

 -Dhttps.protocols=SSLv3,TLSv1,SSLv2Hello

Adding this flag led to the certificate being presented in the javax.net.debug logs, but we were still getting SSLHandshakeException. It seems like the only cipher the server would accept was SSL_RSA_WITH_RC4_128_MD5. This was not picked automatically by our client.

-Dhttps.cipherSuites=SSL_RSA_WITH_RC4_128_MD5

We added this flag to restrict the cipher suits for the client. Together with setting the same restriction programmatically (not sure if both are needed):

socket.setEnabledCipherSuites(new String[] {"SSL_RSA_WITH_RC4_128_MD5"});

Restricting the available cipher suites to the only one that the client could use, made the client pick that cipher suite.

We also did the following changes the the jre/lib/security/java.security file to enable SSLv3 and the SSL_RSA_WITH_RC4_128_MD5 cipher:

• remove SSLv3 from jdk.tls.disabledAlgorithms
• add SSL_RSA_WITH_RC4_128_MD5 to jdk.tls.legacyAlgorithms

This is probably not recommended for production servers, since SSLv3 is obsolete, and the cipher is very old and outdated, but in this case security is not a huge concern (internal application use).

These posts were also helpful to me:

• SSL connection failing for Java 7
• Java 7 (acting as client) SSL handshake failure with keystore and truststore that worked in Java 6
• Received fatal alert: handshake_failure through SSLHandshakeException
• Java cipher suites