Getting error in EVP_OpenInit() of OpenSSL EVP api for RSA decryption, in C

Question

I am facing a problem in RSA decryption using OpenSSL Library (EVP api). Here is my code for key generation

#include 
#include 

        

Answer 1

I see you have used EVP_OpenInit(), EVP_OpenUpdate() and EVP_OpenFinal() for decryption. So I'm going to assume that you've used EVP_SealInit(), EVP_SealUpdate() and EVP_SealFinal() functions for encryption.

So if you have used EVP_SealInit(), you must have passed the address (pointer) to eklen (please check man EVP_SealInit) where the variable gets a value assigned. In analogous to that the eklen variable that you are passing to EVP_OpenInit() must contain a valid key length.

The following is the description from man EVP_OpenInit.

EVP_OpenInit() initializes a cipher context ctx for decryption with cipher type. It decrypts the encrypted symmetric key of length ekl bytes passed in the ek parameter using the private key priv. The IV is supplied in the iv parameter.

I'm not sure here, but I guess you need to use EVP_PKEY_size() function in this case to get the length of the key.

For example (in your decryption code):

....
..
. 
EVP_CIPHER_CTX_init(&ctx); 
ek = malloc( EVP_PKEY_size(pkey)); 

/* Add the following line */
eklen = EVP_PKEY_size(pkey);

if (!EVP_OpenInit(&ctx, EVP_aes_128_cbc(), ek, eklen, iv,pkey)) 
{ 
    fprintf(stderr, "EVP_OpenInit: failed.\n"); 
...
..
.

Answer 2

Are you absolutely sure, that you have initialized variable eklen?