Can RSACryptoServiceProvider (.NET's RSA) use SHA256 for encryption (not signing) instead of SHA1?

Question

When encrypting, can RSACryptoServiceProvider (or any other RSA encryptor available from .NET) use SHA256 instead of SHA1?

SHA1 appears to be hard coded with no way

Answer 1

All of the other answers here are about SIGNING, not ENCRYPTING, with SHA256. I'm going to actually answer the question asked.

Any SHA-2 or higher algorithms are for hashing, not necessarily encrypting/decrypting, but that doesn't mean you can't generate keys using those algorithms, then encrypt/decrypt against them. Technically, I will caveat this for those that would dissent against this answer, this is not "encrypting with SHA256", but it does allow RSA to use a hashed key generated using that algorithm. Let your particular organization decide if that is enough to be NIST/FIPS-compliant, should that be your cause, as it was mine when researching this.

Encryption (using RSA, or another asymmetric encryption algorithm) simply requires a public key (for encrypting) and a private key (for decryption). Once you create keys using that hash, you can encrypt/decrypt against them.

I'm going to piece together some research I did to show a couple of routes you can go for how to get this going using a key created using a SHA-256 hash, that you then encrypt/decrypt against. You can generate a SHA-256 key either by creating a certificate or letting the RSACryptoServiceContainer give you one.

Certificate method

Create your certificate with these lines on a command line:

makecert -r -pe -n "CN=MyCertificate" -a sha256 -b 09/01/2016 -sky exchange C:\Temp\MyCertificate.cer -sv C:\Temp\MyCertificate.pvk
pvk2pfx.exe -pvk C:\Temp\MyCertificate.pvk -pi "MyP@ssw0rd" -spc C:\Temp\MyCertificate.cer -pfx C:\Temp\MyCertificate.pfx -po "MyP@ssw0rd"

Then import the certificate to the local root authority store and use this code:

string input = "test";
string output = string.Empty;

X509Store store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);

X509Certificate2Collection collection = store.Certificates.Find(X509FindType.FindBySubjectName, "MyCertificate", false);

X509Certificate2 certificate = collection[0];

using (RSACryptoServiceProvider cps = (RSACryptoServiceProvider)certificate.PublicKey.Key)
{
    byte[] bytesData = Encoding.UTF8.GetBytes(input);
    byte[] bytesEncrypted = cps.Encrypt(bytesData, false);
    output = Convert.ToBase64String(bytesEncrypted);
}

store.Close();

If you wanted to use SHA512, you would just change that sha256 parameter to sha512 when making the certificate.

Reference: https://social.msdn.microsoft.com/Forums/en-US/69e39ad0-13c2-4b5e-bb1b-972a614813fd/encrypt-with-certificate-sha512?forum=csharpgeneral

Using RSACryptoServiceProvider to generate the keys

private static string privateKey = String.Empty;

private static void generateKeys()
{
    int dwLen = 2048;
    RSACryptoServiceProvider csp = new RSACryptoServiceProvider(dwLen);
    privateKey = csp.ToXmlString(true).Replace("><",">\r\n");
}

public static string Encrypt(string data2Encrypt)
{
    try
    {
        generateKeys();
        RSAx rsax = new RSAx(privateKey, 2048);
        rsax.RSAxHashAlgorithm = RSAxParameters.RSAxHashAlgorithm.SHA256;
        byte[] CT = rsax.Encrypt(Encoding.UTF8.GetBytes(data2Encrypt), false, true); // first bool is for using private key (false forces to use public), 2nd is for using OAEP
        return Convert.ToBase64String(CT);
    }
    catch (Exception ex) 
    { 
        // handle exception
        MessageBox.Show("Error during encryption: " + ex.Message);
        return String.Empty;
    }
}

public static string Decrypt(string data2Decrypt)
{
    try
    {
        RSAx rsax = new RSAx(privateKey, 2048);
        rsax.RSAxHashAlgorithm = RSAxParameters.RSAxHashAlgorithm.SHA256;
        byte[] PT = rsax.Decrypt(Convert.FromBase64String(data2Decrypt), true, true); // first bool is for using private key, 2nd is for using OAEP
        return Encoding.UTF8.GetString(PT);
    }
    catch (Exception ex) 
    { 
        // handle exception
        MessageBox.Show("Error during encryption: " + ex.Message);
        return String.Empty;
    }
}

If you wanted to use SHA512, you could change RSAxHashAlgorithm.SHA256 to RSAxHashAlgorithm.SHA512.

These methods use a DLL called RSAx.DLL, built using the source code at https://www.codeproject.com/Articles/421656/RSA-Library-with-Private-Key-Encryption-in-Csharp , which is not mine (author: Arpan Jati), but I've used it and it is available to the developer community under CodeProject's Open Source License. You could also just bring in 3 classes from that project, instead: RSAx.cs, RSAxParameters.cs, RSAxUtils.cs

The code would take this post over the 30000 char limit, so I'll just post RSAx so you can see what's going on, but all 3 classes are required. You have to change the namespace and reference the System.Numerics assembly.

RSAx.cs

// @Date : 15th July 2012
// @Author : Arpan Jati (arpan4017@yahoo.com; arpan4017@gmail.com)
// @Library : ArpanTECH.RSAx
// @CodeProject: http://www.codeproject.com/Articles/421656/RSA-Library-with-Private-Key-Encryption-in-Csharp  

using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using System.Numerics;
using System.Linq;
using System.Text;
using System.IO;

namespace ArpanTECH
{
    /// 

    /// The main RSAx Class
    /// 
    public class RSAx : IDisposable
    {
        private RSAxParameters rsaParams;
        private RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();

        /// 

        /// Initialize the RSA class.
        /// 
        /// Preallocated RSAxParameters containing the required keys.
        public RSAx(RSAxParameters rsaParams)
        {
            this.rsaParams = rsaParams;
            UseCRTForPublicDecryption = true;
        }

        /// 

        /// Initialize the RSA class from a XML KeyInfo string.
        /// 
        /// XML Containing Key Information
       /// Length of RSA Modulus in bits.
       public RSAx(String keyInfo, int ModulusSize)
       {
            this.rsaParams = RSAxUtils.GetRSAxParameters(keyInfo, ModulusSize);
            UseCRTForPublicDecryption = true;
        }

        /// 

        /// Hash Algorithm to be used for OAEP encoding.
        /// 
        public RSAxParameters.RSAxHashAlgorithm RSAxHashAlgorithm
        {
            set
            {
                rsaParams.HashAlgorithm = value;
            }
        }

        /// 

        /// If True, and if the parameters are available, uses CRT for private key decryption. (Much Faster)
        /// 
        public bool UseCRTForPublicDecryption
        {
            get;  set;
        }

        /// 

        /// Releases all the resources.
        /// 
        public void Dispose()
        {
            rsaParams.Dispose();
        }

        #region PRIVATE FUNCTIONS

        /// 

        /// Low level RSA Process function for use with private key.
        /// Should never be used; Because without padding RSA is vulnerable to attacks.  Use with caution.
        /// 
        /// Data to encrypt. Length must be less than Modulus size in octets.
        /// True to use Private key, else Public.
        /// Encrypted Data
        public byte[] RSAProcess(byte[] PlainText, bool usePrivate)
        {

            if (usePrivate && (!rsaParams.Has_PRIVATE_Info))
            {
                throw new CryptographicException("RSA Process: Incomplete Private Key Info");
            }

            if ((usePrivate == false) && (!rsaParams.Has_PUBLIC_Info))
            {
                throw new CryptographicException("RSA Process: Incomplete Public Key Info");
            }            

            BigInteger _E;
            if (usePrivate)
                _E = rsaParams.D; 
            else
                _E = rsaParams.E;

            BigInteger PT = RSAxUtils.OS2IP(PlainText, false);
            BigInteger M = BigInteger.ModPow(PT, _E, rsaParams.N);

            if (M.Sign == -1)
                return RSAxUtils.I2OSP(M + rsaParams.N, rsaParams.OctetsInModulus, false);            
            else
                return RSAxUtils.I2OSP(M, rsaParams.OctetsInModulus, false);                   
        }

        /// 

        /// Low level RSA Decryption function for use with private key. Uses CRT and is Much faster.
        /// Should never be used; Because without padding RSA is vulnerable to attacks. Use with caution.
        /// 
        /// Data to encrypt. Length must be less than Modulus size in octets.
        /// Encrypted Data
        public byte[] RSADecryptPrivateCRT(byte[] Data)
        {
            if (rsaParams.Has_PRIVATE_Info && rsaParams.HasCRTInfo)
            {
                BigInteger C = RSAxUtils.OS2IP(Data, false);

                BigInteger M1 = BigInteger.ModPow(C, rsaParams.DP, rsaParams.P);
                BigInteger M2 = BigInteger.ModPow(C, rsaParams.DQ, rsaParams.Q);
                BigInteger H = ((M1 - M2) * rsaParams.InverseQ) % rsaParams.P;
                BigInteger M = (M2 + (rsaParams.Q * H));

                if (M.Sign == -1)
                    return RSAxUtils.I2OSP(M + rsaParams.N, rsaParams.OctetsInModulus, false);
                else
                    return RSAxUtils.I2OSP(M, rsaParams.OctetsInModulus, false); 
            }
            else
            {
                throw new CryptographicException("RSA Decrypt CRT: Incomplete Key Info");
            }                             
        }        

        private byte[] RSAProcessEncodePKCS(byte[] Message, bool usePrivate)
        {
            if (Message.Length > rsaParams.OctetsInModulus - 11)
            {
                throw new ArgumentException("Message too long.");
            }
            else
            {
                // RFC3447 : Page 24. [RSAES-PKCS1-V1_5-ENCRYPT ((n, e), M)]
                // EM = 0x00 || 0x02 || PS || 0x00 || Msg 

                List PCKSv15_Msg = new List();

                PCKSv15_Msg.Add(0x00);
                PCKSv15_Msg.Add(0x02);

                int PaddingLength = rsaParams.OctetsInModulus - Message.Length - 3;

                byte[] PS = new byte[PaddingLength];
                rng.GetNonZeroBytes(PS);

                PCKSv15_Msg.AddRange(PS);
                PCKSv15_Msg.Add(0x00);

                PCKSv15_Msg.AddRange(Message);

                return RSAProcess(PCKSv15_Msg.ToArray() ,  usePrivate);
            }
        }

        /// 

        /// Mask Generation Function
        /// 
        /// Initial pseudorandom Seed.
        /// Length of output required.
        /// 
        private byte[] MGF(byte[] Z, int l)
        {
            if (l > (Math.Pow(2, 32)))
            {
                throw new ArgumentException("Mask too long.");
            }
            else
            {
                List result = new List();
                for (int i = 0; i <= l / rsaParams.hLen; i++)
                {
                    List data = new List();
                    data.AddRange(Z);
                    data.AddRange(RSAxUtils.I2OSP(i, 4, false));
                    result.AddRange(rsaParams.ComputeHash(data.ToArray()));
                }

                if (l <= result.Count)
                {
                    return result.GetRange(0, l).ToArray();
                }
                else
                {
                    throw new ArgumentException("Invalid Mask Length.");
                }
            }
        }


        private byte[] RSAProcessEncodeOAEP(byte[] M, byte[] P, bool usePrivate)
        {
            //                           +----------+---------+-------+
            //                      DB = |  lHash   |    PS   |   M   |
            //                           +----------+---------+-------+
            //                                          |
            //                +----------+              V
            //                |   seed   |--> MGF ---> XOR
            //                +----------+              |
            //                      |                   |
            //             +--+     V                   |
            //             |00|    XOR <----- MGF <-----|
            //             +--+     |                   |
            //               |      |                   |
            //               V      V                   V
            //             +--+----------+----------------------------+
            //       EM =  |00|maskedSeed|          maskedDB          |
            //             +--+----------+----------------------------+

            int mLen = M.Length;
            if (mLen > rsaParams.OctetsInModulus - 2 * rsaParams.hLen - 2)
            {
                throw new ArgumentException("Message too long.");
            }
            else
            {
                byte[] PS = new byte[rsaParams.OctetsInModulus - mLen - 2 * rsaParams.hLen - 2];
                //4. pHash = Hash(P),
                byte[] pHash = rsaParams.ComputeHash(P);

                //5. DB = pHash||PS||01||M.
                List _DB = new List();
                _DB.AddRange(pHash);
                _DB.AddRange(PS);
                _DB.Add(0x01);
                _DB.AddRange(M);
                byte[] DB = _DB.ToArray();

                //6. Generate a random octet string seed of length hLen.                
                byte[] seed = new byte[rsaParams.hLen];
                rng.GetBytes(seed);

                //7. dbMask = MGF(seed, k - hLen -1).
                byte[] dbMask = MGF(seed, rsaParams.OctetsInModulus - rsaParams.hLen - 1);

                //8. maskedDB = DB XOR dbMask
                byte[] maskedDB = RSAxUtils.XOR(DB, dbMask);

                //9. seedMask = MGF(maskedDB, hLen)
                byte[] seedMask = MGF(maskedDB, rsaParams.hLen);

                //10. maskedSeed = seed XOR seedMask.
                byte[] maskedSeed = RSAxUtils.XOR(seed, seedMask);

                //11. EM = 0x00 || maskedSeed || maskedDB.
                List result = new List();
                result.Add(0x00);
                result.AddRange(maskedSeed);
                result.AddRange(maskedDB);

                return RSAProcess(result.ToArray(), usePrivate);
            }
        }


        private byte[] Decrypt(byte[] Message, byte [] Parameters, bool usePrivate, bool fOAEP)
        {
            byte[] EM = new byte[0];
            try
            {
                if ((usePrivate == true) && (UseCRTForPublicDecryption) && (rsaParams.HasCRTInfo))
                {
                    EM = RSADecryptPrivateCRT(Message);
                }
                else
                {
                    EM = RSAProcess(Message, usePrivate);
                }
            }
            catch (CryptographicException ex)
            {
                throw new CryptographicException("Exception while Decryption: " + ex.Message);
            }
            catch
            {
                throw new Exception("Exception while Decryption: ");
            }

            try
            {
                if (fOAEP) //DECODE OAEP
                {
                    if ((EM.Length == rsaParams.OctetsInModulus) && (EM.Length > (2 * rsaParams.hLen + 1)))
                    {
                        byte[] maskedSeed;
                        byte[] maskedDB;
                        byte[] pHash = rsaParams.ComputeHash(Parameters);
                        if (EM[0] == 0) // RFC3447 Format : http://tools.ietf.org/html/rfc3447
                        {
                            maskedSeed = EM.ToList().GetRange(1, rsaParams.hLen).ToArray();
                            maskedDB = EM.ToList().GetRange(1 + rsaParams.hLen, EM.Length - rsaParams.hLen - 1).ToArray();
                            byte[] seedMask = MGF(maskedDB, rsaParams.hLen);
                            byte[] seed = RSAxUtils.XOR(maskedSeed, seedMask);
                            byte[] dbMask = MGF(seed, rsaParams.OctetsInModulus - rsaParams.hLen - 1);
                            byte[] DB = RSAxUtils.XOR(maskedDB, dbMask);

                            if (DB.Length >= (rsaParams.hLen + 1))
                            {
                                byte[] _pHash = DB.ToList().GetRange(0, rsaParams.hLen).ToArray();
                                List PS_M = DB.ToList().GetRange(rsaParams.hLen, DB.Length - rsaParams.hLen);
                                int pos = PS_M.IndexOf(0x01);
                                if (pos >= 0 && (pos < PS_M.Count))
                                {
                                    List _01_M = PS_M.GetRange(pos, PS_M.Count - pos);
                                    byte[] M;
                                    if (_01_M.Count > 1)
                                    {
                                        M = _01_M.GetRange(1, _01_M.Count - 1).ToArray();
                                    }
                                    else
                                    {
                                        M = new byte[0];
                                    }
                                    bool success = true;
                                    for (int i = 0; i < rsaParams.hLen; i++)
                                    {
                                         if (_pHash[i] != pHash[i])
                                        {
                                            success = false;
                                            break;
                                        }
                                    }

                                    if (success)
                                    {
                                        return M;
                                    }
                                    else
                                    {
                                        M = new byte[rsaParams.OctetsInModulus]; //Hash Match Failure.
                                        throw new CryptographicException("OAEP Decode Error");
                                    }
                                }
                                else
                                {// #3: Invalid Encoded Message Length.
                                    throw new CryptographicException("OAEP Decode Error");
                                }
                            }
                            else
                            {// #2: Invalid Encoded Message Length.
                                throw new CryptographicException("OAEP Decode Error");
                            }
                        }
                        else // Standard : ftp://ftp.rsasecurity.com/pub/rsalabs/rsa_algorithm/rsa-oaep_spec.pdf
                        {//OAEP : THIS STADNARD IS NOT IMPLEMENTED
                            throw new CryptographicException("OAEP Decode Error");
                        }
                    }
                    else
                    {// #1: Invalid Encoded Message Length.
                        throw new CryptographicException("OAEP Decode Error");
                    }
                }
                else // DECODE PKCS v1.5
                {
                    if (EM.Length >= 11)
                    {
                        if ((EM[0] == 0x00) && (EM[1] == 0x02))
                        {
                            int startIndex = 2;
                            List PS = new List();
                            for (int i = startIndex; i < EM.Length; i++)
                            {
                                if (EM[i] != 0)
                                {
                                    PS.Add(EM[i]);
                                }
                                else
                                {
                                    break;
                                }
                            }

                            if (PS.Count >= 8)
                            {
                                int DecodedDataIndex = startIndex + PS.Count + 1;
                                if (DecodedDataIndex < (EM.Length - 1))
                                {
                                    List DATA = new List();
                                    for (int i = DecodedDataIndex; i < EM.Length; i++)
                                    {
                                        DATA.Add(EM[i]);
                                    }
                                    return DATA.ToArray();
                                }
                                else
                                {
                                    return new byte[0];
                                    //throw new CryptographicException("PKCS v1.5 Decode Error #4: No Data");
                                }
                            }
                            else
                            {// #3: Invalid Key / Invalid Random Data Length
                                throw new CryptographicException("PKCS v1.5 Decode Error");
                            }
                        }
                        else
                        {// #2: Invalid Key / Invalid Identifiers
                            throw new CryptographicException("PKCS v1.5 Decode Error");
                        }
                    }
                    else
                    {// #1: Invalid Key / PKCS Encoding
                        throw new CryptographicException("PKCS v1.5 Decode Error");
                    }

                }
            }
            catch (CryptographicException ex)
            {
                throw new CryptographicException("Exception while decoding: " + ex.Message);
            }
            catch
            {
                throw new CryptographicException("Exception while decoding");
            }


        }

        #endregion

        #region PUBLIC FUNCTIONS

        /// 

        /// Encrypts the given message with RSA, performs OAEP Encoding.
        /// 
        /// Message to Encrypt. Maximum message length is (ModulusLengthInOctets - 2 * HashLengthInOctets - 2)
        /// Optional OAEP parameters. Normally Empty. But, must match the parameters while decryption.
        /// True to use Private key for encryption. False to use Public key.
        /// Encrypted message.
        public byte[] Encrypt(byte[] Message, byte[] OAEP_Params, bool usePrivate)
        {
            return RSAProcessEncodeOAEP(Message, OAEP_Params, usePrivate);
        }

        /// 

        /// Encrypts the given message with RSA.
        /// 
        /// Message to Encrypt. Maximum message length is For OAEP [ModulusLengthInOctets - (2 * HashLengthInOctets) - 2] and for PKCS [ModulusLengthInOctets - 11]
        /// True to use Private key for encryption. False to use Public key.
        /// True to use OAEP encoding (Recommended), False to use PKCS v1.5 Padding.
        /// Encrypted message.
        public byte[] Encrypt(byte[] Message, bool usePrivate, bool fOAEP)
        {
            if (fOAEP)
            {
                return RSAProcessEncodeOAEP(Message, new byte[0], usePrivate);
            }
            else
            {
                return RSAProcessEncodePKCS(Message, usePrivate);
            }
        }

        /// 

        /// Encrypts the given message using RSA Public Key.
        /// 
        /// Message to Encrypt. Maximum message length is For OAEP [ModulusLengthInOctets - (2 * HashLengthInOctets) - 2] and for PKCS [ModulusLengthInOctets - 11]
        /// True to use OAEP encoding (Recommended), False to use PKCS v1.5 Padding.
        /// Encrypted message.
        public byte[] Encrypt(byte[] Message,  bool fOAEP)
        {
            if (fOAEP)
            {
                return RSAProcessEncodeOAEP(Message, new byte[0], false);
            }
            else
            {
                return RSAProcessEncodePKCS(Message, false);
            }
        }

        /// 

        /// Decrypts the given RSA encrypted message.
        /// 
        /// The encrypted message.
        /// True to use Private key for decryption. False to use Public key.
        /// True to use OAEP.
        /// Encrypted byte array.
        public byte[] Decrypt(byte[] Message, bool usePrivate, bool fOAEP)
        {
            return Decrypt(Message, new byte[0], usePrivate, fOAEP);
        }

        /// 

        /// Decrypts the given RSA encrypted message.
        /// 
        /// The encrypted message.
        /// Parameters to the OAEP algorithm (Must match the parameter while Encryption).
        /// True to use Private key for decryption. False to use Public key.
        /// Decrypted byte array.
        public byte[] Decrypt(byte[] Message, byte[] OAEP_Params, bool usePrivate)
        {
            return Decrypt(Message, OAEP_Params, usePrivate, true);
        }

        /// 

        /// Decrypts the given RSA encrypted message using Private key.
        /// 
        /// The encrypted message.
        /// True to use OAEP.
        /// Decrypted byte array.
        public byte[] Decrypt(byte[] Message,  bool fOAEP)
        {
            return Decrypt(Message, new byte[0], true, fOAEP);
        }
        #endregion
    }
}