Encrypting large streams to be sent via Amazon S3

问题

I want to encrypt stream and then send it using Amazon S3. I'm using legacy code and have two important parameters: non-encrypted InputStream and its length. This is important as AmazonS3Client wants to know the length of the stream before it uploads it.

Encrypting a stream is not very difficult task:

        InputStream in = new FileInputStream("path-to-file");
        KeyGenerator keygen = KeyGenerator.getInstance("AES");
        Key key = keygen.generateKey();

        Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
        cipher.init(Cipher.ENCRYPT_MODE, key);

        OutputStream encryptedStream = new ByteArrayOutputStream();
        CipherOutputStream out = new CipherOutputStream(encryptedStream, cipher);
        out.write(in.read());
        out.flush();

        byte[] bytes = ((ByteArrayOutputStream) encryptedStream).toByteArray();
        InputStream encryptedInput = new ByteArrayInputStream(bytes);

However, this approach could be used for small files only as their are written to a byte array held in memory. The thing is that I'd like to encrypt big files (> 1TB). How do I do this? More specifically, I get InputStream and my job is to return encrypted InputStream that will be used by Amazon S3.

回答1:

do not use ByteArrayOutputStream

    InputStream in = new FileInputStream("path-to-file");
    KeyGenerator keygen = KeyGenerator.getInstance("AES");
    Key key = keygen.generateKey();

    Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
    cipher.init(Cipher.ENCRYPT_MODE, key);

    try(CipherOutputStream out = new CipherOutputStream(
            new FileOutputStream(new File("path to enc file")), cipher))
    {
        byte[] buffer = new byte[8192];
        int count;
        while ((count = in.read(buffer)) > 0)
        {
            out.write(buffer, 0, count);
        }
    }

    return new FileInputStream(new File("path to enc file"));

One more answer with piped streams.

    InputStream in = new FileInputStream("path-to-file");
    KeyGenerator keygen = KeyGenerator.getInstance("AES");
    Key key = keygen.generateKey();

    Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
    cipher.init(Cipher.ENCRYPT_MODE, key);

    PipedOutputStream pout = new PipedOutputStream();
    //write data in a separate thread.
    new Thread(() -> {
        try(CipherOutputStream out = new CipherOutputStream(
                pout, cipher))
        {
            byte[] buffer = new byte[8192];
            int count;
            while ((count = in.read(buffer)) > 0)
            {
                out.write(buffer, 0, count);
            }
        } catch (IOException  e)
        {
            e.printStackTrace();
        }
    }).start();

    return new PipedInputStream(pout);

来源：https://stackoverflow.com/questions/51117639/encrypting-large-streams-to-be-sent-via-amazon-s3

标签

java

encryption

amazon-s3