I need some help using OpenSSL to generate a signature of a block of data using C (Windows and Linux). The application has to do with Google authentication. The instruction
You can use RSA_sign to sign the data with SHA256 hash. You can call this
RSA_sign(NID_sha256, digest, digest_len, &sign_buffer, sign_len, rsa_key);
You have calculate SHA256 hash of the data into digest buffer. rsa_key should be initialized.
Since, you have .p12 file, you need to do the following.
Create PKCS12 structure from .p12 file.
PKCS12 * p12 = d2i_PKCS12_fp(fp_to_p12_file, NULL);
Parse the p12 structure to certificate and key.
PKCS12_parse(p12, passphrase, &evp_pkey, &x509_cert, NULL);
It will give EVP_PKEY structure from which you can get RSA structure using EVP_PKEY_get1_RSA or access evp_pkey->pkey.rsa member of the structure.
This should help you to start.