What's wrong with IBM's JCE provider?

Question

I have a JCE test that works fine with all Sun JDKs I have tried, but fails with various IBM J9 JDKs (e.g. 1.6.0 build pwi3260sr8-20100409_01(SR8)). The exception below happ

Answer 1

@Stephen C / @FelixM: IBM seems to be completely clueless about how RSA cryptography works and how it is intended to be used. Basically both operations (encrypt / decrypt) must be available for the public AND private key.

Encrypt with public key is needed to transmit the client-side part of the pre master secret in SSL/TLS handshakes. The server needs to decrypt with its private key. But if they negotiate something like ECDHE_RSA the server needs to SIGN parts of the handshake with the private key - thats encrypt with PrivateKey. Vice versa the client needs to decrypt with the public key from the certificate of the server to verify the hash value of the signature. (proving authenticity of the message)

So if I try to run ECDHE_RSA (server-side) on latest IBM JDK 7 the following happens:

java.security.InvalidKeyException: Private key cannot be used to encrypt.
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
   at java.lang.Thread.run(Thread.java:777)
   at com.ibm.crypto.provider.RSASSL.engineInit(Unknown Source)
   at javax.crypto.Cipher.init(Unknown Source)
   at javax.crypto.Cipher.init(Unknown Source)
   at java.security.Signature$CipherAdapter.engineInitSign(Signature.java:1239)
   at java.security.Signature$Delegate.init(Signature.java:1116)
   at java.security.Signature$Delegate.chooseProvider(Signature.java:1076)
   at java.security.Signature$Delegate.engineInitSign(Signature.java:1140)
   at java.security.Signature.initSign(Signature.java:522)
   at net.vx4.lib.tls.core.TLSSignature.createSignature(TLSSignature.java:120)

As you can see we're using "Signature" and call "initSign", which requires indeed a PrivateKey. This proves IBM being clueless about this fact and obviously they don't even have valid regression tests!

Use another crypto provider and don't believe IBM until they change their mind.

Best regards, Christian