Can you help me get my head around openssl public key encryption with rsa.h in c++?

Question

I am trying to get my head around public key encryption using the openssl implementation of rsa in C++. Can you help? So far these are my thoughts (please do correct if nece

Answer 1

Thanks @Caf. Your post helped. However I got

The program '[7056] Encryption2.exe: Native' has exited with code -1073741811 (0xc000000d) for the line

  PEM_read_RSA_PUBKEY(rsa_pkey_file, &rsa_pkey, NULL, NULL)

I changed to

BIO *bio;
X509 *certificate;

bio = BIO_new(BIO_s_mem());
BIO_puts(bio, (const char*)data);
certificate = PEM_read_bio_X509(bio, NULL, NULL, NULL);
EVP_PKEY *pubkey = X509_get_pubkey (certificate);
rsa_pkey = EVP_PKEY_get1_RSA(pubkey);

Where data has the PEM file with only public key. My challenge was to encrypt in C++ and decrypt in java. I transmitted the base64 encoded ek of size eklen (i did not use eklen_n) and decrypted to get the AES key using the RSA private key. Then I decrypted the cipher file using this AES key. It worked fine.