M2crypto signature “algorithm”

Question

These two codes provide the same signature, which is expected:

code1:

from M2Crypto import RSA, EVP
import base64, hashlib

text = "some text"

pkey = EVP.load_key("mykey.pem")  #"mykey.pem" was generated as: openssl genrsa -des3 -out mykey.pem 2048
pkey.sign_init()
pkey.sign_update(text)
signature = pkey.sign_final()
print base64.b64encode(signature)

code2:

pkey = RSA.load_key("mykey.pem")
signature = pkey.sign(hashlib.sha1(text).digest())
print base64.b64encode(signature)

However, if I want to "imitate" the signature algorithm, i.e. encrypting the digest with the private key, I get a different signature, i.e.:

pkey = RSA.load_key("mykey.pem")
signature = pkey.private_encrypt(hashlib.sha1(text).digest(), RSA.pkcs1_padding)
print base64.b64encode(signature)  #different from the two above

Could you please provide some explanation? What is wrong with the latter way of signing?

Answer 1

I believe the difference is that RSA_sign signs the digest PKCS1 algorithmIdentifier along with the digest data, where RSA_private_encrypt signs only the digest data.

From the RSA_private_encrypt man page:

RSA_PKCS1_PADDING
    PKCS #1 v1.5 padding. This function does not handle the
    algorithmIdentifier specified in PKCS #1. When generating or
    verifying PKCS #1 signatures, RSA_sign(3) and RSA_verify(3) should
    be used.

Answer 2

What happens internally in EVP.sign() is as follows (as opposed to plain RSA.sign()):

sha1_hash = hashlib.sha1(MESSAGE).digest()
# Add ASN.1 SHA-1 OID prefix
sha1_asn1_prefix = '3021300906052b0e03021a05000414'.decode('hex')
asn1_hash = sha1_asn1_prefix + sha1_hash
rsa = RSA.load_key(KEY)
# Use PKCS#1 padding
signature = rsa.private_encrypt(asn1_hash, RSA.pkcs1_padding).encode('hex')

See this answer for longer explanation and this gist for a full example.

But the bottom line is that EVP.sign() should be used instead as in code 1 above - it does the right thing internally.