How to encrypt a large file in openssl using public key

Question

How can I encrypt a large file with a public key so that no one other than who has the private key be able to decrypt it?

I can make RSA public and private keys but

Answer 1

Encrypting a very large file using smime is not advised since you might be able to encrypt large files using the -stream option, but not decrypt the resulting file due to hardware limitations see: problem decrypting big files

As mentioned above Public-key crypto is not for encrypting arbitrarily long files. Therefore the following commands will generate a pass phrase, encrypt the file using symmetric encryption and then encrypt the pass phrase using the asymmetric (public key). Note: the smime includes the use of a primary public key and a backup key to encrypt the pass phrase. A backup public/private key pair would be prudent.

Random Password Generation

Set up the RANDFILE value to a file accessible by the current user, generate the passwd.txt file and clean up the settings

export OLD_RANDFILE=$RANDFILE
RANDFILE=~/rand1
openssl rand -base64 2048 > passwd.txt
rm ~/rand1
export RANDFILE=$OLD_RANDFILE

Encryption

Use the commands below to encrypt the file using the passwd.txt contents as the password and AES256 to a base64 (-a option) file. Encrypt the passwd.txt using asymetric encryption into the file XXLarge.crypt.pass using a primary public key and a backup key.

openssl enc -aes-256-cbc -a -salt -in XXLarge.data -out XXLarge.crypt -pass file:passwd.txt
openssl smime -encrypt -binary -in passwd.txt -out XXLarge.crypt.pass -aes256 PublicKey1.pem PublicBackupKey.pem
rm passwd.txt

Decryption

Decryption simply decrypts the XXLarge.crypt.pass to passwd.tmp, decrypts the XXLarge.crypt to XXLarge2.data, and deletes the passwd.tmp file.

openssl smime -decrypt -binary -in XXLarge.crypt.pass -out passwd.tmp -aes256 -recip PublicKey1.pem -inkey PublicKey1.key
openssl enc -d -aes-256-cbc -a -in XXLarge.crypt -out XXLarge2.data -pass file:passwd.tmp
rm passwd.tmp

This has been tested against >5GB files..

5365295400 Nov 17 10:07 XXLarge.data
7265504220 Nov 17 10:03 XXLarge.crypt
      5673 Nov 17 10:03 XXLarge.crypt.pass
5365295400 Nov 17 10:07 XXLarge2.data

Answer 2

In more explanation for n. 'pronouns' m.'s answer,

Public-key crypto is not for encrypting arbitrarily long files. One uses a symmetric cipher (say AES) to do the normal encryption. Each time a new random symmetric key is generated, used, and then encrypted with the RSA cipher (public key). The ciphertext together with the encrypted symmetric key is transferred to the recipient. The recipient decrypts the symmetric key using his private key, and then uses the symmetric key to decrypt the message.

There is the flow of Encryption:

+---------------------+      +--------------------+
|                     |      |                    |
| generate random key |      |   the large file   |
|        (R)          |      |        (F)         |
|                     |      |                    |
+--------+--------+---+      +----------+---------+
         |        |                     |
         |        +------------------+  |
         |                           |  |
         v                           v  v
+--------+------------+     +--------+--+------------+
|                     |     |                        |
| encrypt (R) with    |     | encrypt (F)            |
| your RSA public key |     | with symmetric key (R) |
|                     |     |                        |
|  ASym(PublicKey, R) |     |     EF = Sym(F, R)     |
|                     |     |                        |
+----------+----------+     +------------+-----------+
           |                             |
           +------------+ +--------------+
                        | |
                        v v
         +--------------+-+---------------+
         |                                |
         |   send this files to the peer  |
         |                                |
         |     ASym(PublicKey, R) + EF    |
         |                                |
         +--------------------------------+

And the flow of Decryption:

   +----------------+        +--------------------+
   |                |        |                    |
   | EF = Sym(F, R) |        | ASym(PublicKey, R) |
   |                |        |                    |
   +-----+----------+        +---------+----------+
         |                             |
         |                             |
         |                             v
         |   +-------------------------+-----------------+
         |   |                                           |
         |   |             restore key (R)               |
         |   |                                           |
         |   | R <= ASym(PrivateKey, ASym(PublicKey, R)) |
         |   |                                           |
         |   +---------------------+---------------------+
         |                         |
         v                         v
     +---+-------------------------+---+
     |                                 |
     |       restore the file (F)      |
     |                                 |
     |      F <= Sym(Sym(F, R), R)     |
     |                                 |
     +---------------------------------+

Besides, you can use this commands:

# generate random symmetric key
openssl rand -base64 32 > /config/key.bin

# encryption
openssl rsautl -encrypt -pubin -inkey /config/public_key.pem -in /config/key.bin -out /config/key.bin.enc
openssl aes-256-cbc -a -pbkdf2 -salt -in  $file_name -out $file_name.enc -k $(cat /config/key.bin)

# now you can send this files: $file_name.enc + /config/key.bin.enc

# decryption
openssl rsautl -decrypt -inkey /config/private_key.pem -in /config/key.bin.enc -out /config/key.bin
openssl aes-256-cbc -d -a -in $file_name.enc -out $file_name -k $(cat /config/key.bin)